Table of Contents

1. Machine Learning Basics

2. Machine Learning Application

3. Machine Learning Concept

4. Mathematics for Machine Learning

5. Machine Learning Algorithm

---

Machine Learning Basics

 

Learning:

The ability to predict or assign a label to a new observation from a past experience. Suppose you want to prepare Maggie for 3 people.
The steps you follow are like:

• Boil water & chop the veggies.
• Cook Maggi and veggies in a separate pan.
• Add spices.
• Garnish and enjoy.

Preparing any dish requires basic actions such as frying, cutting, etc. In the recipe set of actions followed in which order the ingredient should be carried out.

A software library is just like a cookbook. We have been so developed in terms of technology in the last few years that computer and digital technology are so widely used now.

Still, we have lots of problems to solve, it is unsolved like predicting custom behaviour is one, differentiating spam from normal ones.

machine learning is not just a database or programming problem, but also a requirement for AI.

Our behaviour always depends on the environment some behaviour we learn phase by phase, some behaviour we have inbuilt, and some are created unconsciously by our mind. The evaluation gave us a large brain and a mechanism that we could update ourselves with experience and adapt to a different environment. That is why human beings have survived.

When we learn the best strategy in a certain situation that knowledge is stored in our brain, and when the situation is we recall the suitable strategy and act accordingly.

each of us, actually every animal is a data scientist, we collect information about our environment using our sensors and then we process the data to devise rules of behaviour to control our actions in different circumstances to minimize pain and maximize pleasure. We store everything in memory then we recall and use them when needed.
There are certain limits to learning we can’t learn everything without the Limited capacity of our brain.

Understanding the brain:  

According to Marr 1982, understanding an information processing system works at three levels of analysis,

• Computational theory 
• Representation and algorithm 
• Three hydro and implementation 

Going in the opposite direction from bottom to top is called reverse engineering.

We are also a decision Civilization, we make every day many decisions directly or indirectly, like

• Which video to show next on YouTube,
• Which coupon the customer responds to the 
• Which position should a page/AD be shown for a query?

 

Randomness and Probability  

You must have play Ludo once in your life, if you remember there are six possible outcomes 1 2 3 4 5 6.

When you throw the dice, you don’t know the exact outcomes, it can be anything from 6 possibilities. But we can say something about the probability. when you don’t know those properties and want to estimate them then we use statistics.

• Each data instance is called an example.
• Collections of such examples are called samples.

The aim is to build a model to explain the process that generates the sample. Sometimes it feels like gambling for some people.

Model Selection:

MODEL: It is a template of the relationship between the inputs and the outputs. Assume we have different laptops with different Ram.

Index	RAM	PRICE
1	2 GB	25000
2	4 GB	30000
3	8 GB	40000
4	16 GB	50000
5	32 GB	80000
6	64 GB	90000
7	128 GB	150000

 

We have seven laptops whose Ram and prices are recorded. The Index is just to name them; the order of the laptop is not important. 
y= 30000 – 0.5 900x 
Here y=estimated price, ₹30,000 is the avg. laptop cost. Additional for 1GB RAM decreases by ₹5900.

The features of cars we take into consideration are called parameters.
Model selection: the task of choosing between models.

The above example we have taken on the basis of the population of all possible laptops in the population and recorded the RAM and price of all of them.

The above one is a small subset which is our sample. From the statistical point of view, we don’t particularly care about this sample, there can be mistakes in it, or it could just have been a different sample, what we care about is the population from which any sample is drawn.

 

Machine Learning Application

Machine learning is a subset of artificial intelligence (AI) that involves training algorithms to make predictions or take actions based on data. These algorithms can learn from data without being explicitly programmed and can improve their performance over time. There are many different applications of machine learning, including image and speech recognition, natural language processing, and predictive analytics. Machine learning algorithms can be used to analyze large datasets and make predictions or recommendations based on that data. For example, a machine learning algorithm might be trained to recognize images of dogs, and then be able to identify dogs in new images that it has not seen before. Machine learning is being used in a wide range of industries, including healthcare, finance, and e-commerce, to improve decision-making and automate processes.

Machine Learning Paradigms

• Supervised Learning
• Un-Supervised Learning
• Semi-Supervised learning
• Active Learning
• Reinforcement Learning

Supervised Learning

• REGRESSION:
  The task of estimating or predicting a numeric value is called regression in statistics.
  • Price Prediction
  • Demand forecasting
  • Supply forecasting
  • Count prediction
  • Sales and Reverse Prediction
• CLASSIFICATION:
  The task of estimating or predicting a categorical class is called classification.
  • Email Classification
  • Email Spam detection
  • Video Activity Recognition
  • Blog Post activity recognition
  • Pixel Classification
• RECOMMENDATION:
  The task of estimating or predicting user preference from a large pool of options is called a recommendation.
  • On Netflix, After watching a crime series, it suggests other similar series to the user.
  • On Youtube, After watching any video, it suggests similar types of videos.
• RETRIEVAL: The Task of predicting the relevance of an entity to a “quality” is called Retrieval.
  • When we search on Google, Night Clubs in Navy Mumbai, It shows the relative and needed information on SERP.

The aim of machine learning is mainly to replicate the training data. But the correct prediction of new cases.
How well a model trained on the training set predicts the right output for such a new instance is called the generalization ability of the model and the learning algorithm.

Learning or fitting a model to data is an “ill -Posed Problem”

Every learning algorithm makes a set of assumptions about the data to find a unique model and the set of assumptions is called the induction bias of the learning algorithm (Mitchell 1997).

Un-Supervised Learning

Unsupervised learning is a type of machine learning algorithm in which the model is not trained on labeled data. Instead, the model is trained on a dataset that is not labeled, and it must find patterns and relationships in the data on its own. This means that the model must learn to identify and extract useful features from the data without any guidance or supervision.

One of the main goals of unsupervised learning is to identify patterns and structures in the data that can be used to group similar data points together. This is often referred to as clustering. For example, a clustering algorithm might be trained on a dataset containing customer data, and it might learn to group customers together based on their spending habits, location, age, or other factors.

Another common application of unsupervised learning is dimensionality reduction. This is the process of reducing the number of features or dimensions in a dataset while still preserving as much of the relevant information as possible. This can be useful for visualizing high-dimensional data, or for speeding up the training of other machine learning algorithms.

Unsupervised learning algorithms can be divided into two main categories: density-based and partition-based. Density-based algorithms attempt to identify clusters of data points that are densely packed together, while partition-based algorithms divide the data into a predetermined number of clusters.

Some examples of unsupervised learning algorithms include k-means clustering, hierarchical clustering, and principal component analysis (PCA). These algorithms have been widely used in a variety of applications, including customer segmentation, anomaly detection, and image compression.

Overall, unsupervised learning is a powerful tool for discovering hidden patterns and structures in data. By allowing the model to learn without supervision, unsupervised learning algorithms can uncover insights and relationships that might not have been apparent using other methods.

Here is a list of some common types of unsupervised learning algorithms:

• Density-based algorithms: DBSCAN, OPTICS
• Partition-based algorithms: k-means, Fuzzy c-means, Expectation-maximization (EM)
• Hierarchical algorithms: Agglomerative clustering, Divisive clustering
• Dimensionality reduction algorithms: Principal component analysis (PCA), Singular value decomposition (SVD), t-distributed stochastic neighbor embedding (t-SNE)
• Generative algorithms: Restricted Boltzmann machines (RBM), Generative adversarial networks (GAN)
• Anomaly detection algorithms: One-class SVM, Isolation Forest

These are just a few examples of the many different types of unsupervised learning algorithms that are available. Many other algorithms have been developed and are being actively researched in the field of machine learning.

 

Unsupervised learning has many practical applications in a variety of fields. Some examples include:

• Customer segmentation: Unsupervised learning algorithms can be used to group customers into different segments based on their behavior, such as their spending habits, location, or other factors. This can be useful for targeted marketing or personalized recommendations.
• Anomaly detection: Unsupervised learning algorithms can be used to identify unusual or unexpected patterns in data, such as fraudulent transactions or malfunctioning equipment.
• Image compression: Dimensionality reduction algorithms, such as PCA, can be used to reduce the number of features in an image while still preserving its essential information. This can be useful for reducing the size of an image file without significantly affecting its quality.
• Recommendation systems: Unsupervised learning algorithms can be used to identify similar items, such as products or articles, based on user behavior or other factors. This can be useful for making personalized recommendations to users.
• Natural language processing: Unsupervised learning algorithms can be used to group words or sentences into similar categories, or to identify topics or themes in a large corpus of text. This can be useful for tasks such as sentiment analysis or automatic summarization.

These are just a few examples of the many ways in which unsupervised learning can be used in practice. As the field of machine learning continues to advance, it is likely that new and innovative applications of unsupervised learning will be developed.

Semi-Supervised learning

Semi-supervised learning is a type of machine learning algorithm that uses both labeled and unlabeled data to train a model. This can be useful when there is a large amount of unlabeled data available, but only a small amount of labeled data.

In semi-supervised learning, the model is first trained on the labeled data using a supervised learning algorithm. This provides the model with some initial knowledge about the relationships and patterns in the data. The model is then fine-tuned using the unlabeled data, which allows it to learn additional information and improve its performance.

One of the main advantages of semi-supervised learning is that it can make use of a larger amount of data than supervised learning, which can lead to better performance. This can be especially useful when labeled data is scarce or expensive to obtain.

Some examples of semi-supervised learning algorithms include self-training, co-training, and multi-view learning. These algorithms have been used in a variety of applications, including image classification, speech recognition, and natural language processing.

Overall, semi-supervised learning is a useful tool for improving the performance of machine learning models when only a small amount of labeled data is available. By combining the strengths of supervised and unsupervised learning, semi-supervised learning algorithms can learn more from the data and achieve better results.

 

There are many different types of semi-supervised learning algorithms, and new algorithms are being developed and researched all the time. Here are some examples of common semi-supervised learning algorithms:

• Self-training: In self-training, a supervised learning algorithm is trained on the labeled data, and then it is used to label the unlabeled data. The model is then retrained on the combined labeled and unlabeled data. This can be useful for improving the performance of the model by incorporating additional information from the unlabeled data.
• Co-training: In co-training, the labeled and unlabeled data are split into two different views. Two separate supervised learning models are trained on each view, and then the models are combined to make predictions on the unlabeled data. This can be useful for leveraging complementary information from different views of the data.
• Multi-view learning: In multi-view learning, the data is represented in multiple different views or modalities, such as text and images. A semi-supervised learning algorithm is used to learn from all of the views simultaneously, and to make predictions based on the combined information.
• Generative adversarial networks (GANs): GANs are a type of semi-supervised learning algorithm that uses two separate models: a generator, which generates synthetic data, and a discriminator, which tries to identify whether a data point is real or synthetic. The generator is trained to produce data that is indistinguishable from the real data, while the discriminator is trained to identify real data. This can be useful for improving the performance of a model by training it on a larger dataset of synthetic data.
• Ladder networks: Ladder networks are a type of semi-supervised learning algorithm. They are a variation of the popular autoencoder architecture, which is a type of neural network used for unsupervised learning.
• Pseudo-labeling: Pseudo-labeling is a semi-supervised learning technique in which a trained model is used to label a large amount of unlabeled data. The labeled data is then combined with the original labeled data and used to retrain the model. This can be useful for improving the performance of the model by incorporating additional information from the unlabeled data.
• Tri-training: Tri-training is a semi-supervised learning technique that uses three different models to make predictions on unlabeled data. The models are trained on the labeled data in a supervised manner, and then they are used to make predictions on the unlabeled data. The predicted labels are then combined and used to train a final model.
• SSL with small-loss trick: SSL with small-loss trick is a semi-supervised learning technique that uses a small amount of labeled data to train a model, and then uses the model’s predictions on unlabeled data to improve its performance. The technique is based on the idea that a model’s predictions on unlabeled data are likely to be correct if the model has a small loss on the labeled data.
• Deep co-training: Deep co-training is a semi-supervised learning technique that uses multiple deep learning models to make predictions on unlabeled data. The models are trained on different views of the data, and they are used to label the unlabeled data. The labeled data is then used to train a final deep learning model.

These are just a few examples of the many different types of semi-supervised learning algorithms that are available. There are many other algorithms that have been developed and are being actively researched in the field of machine learning.

Semi-Supervised learning has many potential practical applications, including:

• Image classification: Semi-supervised learning has been used to train image classification models with a small amount of labeled data and a large amount of unlabeled data. By incorporating information from the unlabeled data, the model can improve its performance on tasks such as identifying objects in images and recognizing scenes.
• Sentiment analysis: Semi-supervised learning has been used to train sentiment analysis models with a small amount of labeled data and a large amount of unlabeled data. By incorporating information from the unlabeled data, the model can improve its performance on tasks such as identifying the sentiment of a text and detecting irony.
• Fraud detection: Semi-supervised learning has been used to train fraud detection models with a small amount of labeled data and a large amount of unlabeled data. By incorporating information from the unlabeled data, the model can improve its performance on tasks such as identifying fraudulent transactions and detecting patterns of fraudulent behavior.
• Speech recognition: Semi-supervised learning has been used to train speech recognition models with a small amount of labeled data and a large amount of unlabeled data. By incorporating information from the unlabeled data, the model can improve its performance on tasks such as transcribing speech and recognizing spoken commands.
• Natural language processing: Semi-supervised learning has been used to train natural language processing models with a small amount of labeled data and a large amount of unlabeled data. By incorporating information from the unlabeled data, the model can improve its performance on tasks such as part-of-speech tagging and named entity recognition.

These are just a few examples of the many potential practical applications of semi-supervised learning. As research in the field continues to advance, it is likely that new and exciting applications of semi-supervised learning will be developed.

 

 

Active Learning

Active learning is a type of machine learning in which the model is able to interact with the environment and request labels for specific data points. This allows the model to actively select the data that it wants to be labeled, rather than being passively trained on a fixed dataset.

The main advantage of active learning is that it can be more efficient than traditional supervised learning. By selecting the most informative data points to be labeled, the model can learn more quickly and with less labeled data. This can be especially useful when labeling data is time-consuming or expensive.

There are many different active learning algorithms, and the specific algorithm that is best suited for a particular task will depend on the data and the desired outcome. Some examples of active learning algorithms include uncertainty sampling, expected error reduction, and query by committee.

Active learning has been applied to a variety of tasks, including image classification, speech recognition, and natural language processing. It has shown promising results in comparison to traditional supervised learning, and it is an active area of research in the field of machine learning.

 

There are many different active learning algorithms, and the specific algorithm that is best suited for a particular task will depend on the data and the desired outcome. Here are some examples of common active learning algorithms:

• Uncertainty sampling: In uncertainty sampling, the model selects the data points that it is most uncertain about to be labeled. This can be useful for reducing the model’s uncertainty and improving its performance.
• Expected error reduction: In expected error reduction, the model selects the data points that are expected to have the greatest impact on the model’s performance if they are labeled. This can be useful for maximizing the model’s improvement with each labeled data point.
• Query by committee: In query by committee, a committee of models is trained on the labeled data, and the data points that are disagreed upon by the models are selected to be labeled. This can be useful for reducing the disagreement among the models and improving their performance.
• Query by divergence: In query by divergence, the model selects the data points that are most different from the existing labeled data to be labeled. This can be useful for improving the model’s ability to generalize to new data.
• Query by uncertainty and density: In query by uncertainty and density, the model selects data points that are both uncertain and dense to be labeled. This can be useful for reducing the model’s uncertainty and improving its performance on dense data.

Active learning has many potential practical applications, including:

• Text classification: Active learning has been used to train text classification models with a small amount of labeled data. By selecting the most informative data points to be labeled, the model can learn more quickly and with less labeled data, improving its performance on tasks such as sentiment analysis and topic classification.
• Medical diagnosis: Active learning has been used to train medical diagnosis models with a limited amount of labeled data. By selecting the most informative data points to be labeled, the model can improve its performance on tasks such as identifying diseases and predicting patient outcomes.
• Image annotation: Active learning has been used to train image annotation models with a small amount of labeled data. By selecting the most informative images to be labeled, the model can improve its performance on tasks such as object detection and image segmentation.
• Drug discovery: Active learning has been used to train models for drug discovery with a limited amount of labeled data. By selecting the most informative data points to be labeled, the model can improve its performance on tasks such as identifying potential drug candidates and predicting their effectiveness.
• Natural language generation: Active learning has been used to train natural language generation models with a small amount of labeled data. By selecting the most informative data points to be labeled, the model can improve its performance on tasks such as generating coherent and grammatically correct sentences.

These are just a few examples of the many potential practical applications of active learning. As research in the field continues to advance, it is likely that new and exciting applications of active learning will be developed.

 

 

 

Reinforcement Learning

Reinforcement learning is a type of machine learning in which an agent learns to take actions in an environment in order to maximize a reward. The agent receives feedback in the form of rewards and punishments, and it uses this feedback to improve its performance over time.

In reinforcement learning, the agent learns by trial and error, trying different actions and observing the resulting rewards. This is different from supervised learning, in which the agent is trained on a dataset of labeled examples, and from unsupervised learning, in which the agent learns from unlabeled data.

The main advantage of reinforcement learning is that it allows the agent to learn from its interactions with the environment, rather than from pre-labeled data. This makes it well-suited for problems in which it is difficult or expensive to obtain labeled data, or in which the environment is changing over time.

There are many different reinforcement learning algorithms, and the specific algorithm that is best suited for a particular task will depend on the environment and the desired outcome. Here are some examples of common reinforcement learning algorithms:

• Q-learning: Q-learning is a model-free reinforcement learning algorithm that learns a action-value function, which estimates the expected future reward for each action in a given state. The agent selects the action with the highest expected reward, and the action-value function is updated based on the observed reward.
• Monte Carlo tree search: Monte Carlo tree search is a model-based reinforcement learning algorithm that uses a search tree to explore the possible actions and rewards in the environment. The algorithm simulates the future actions and rewards based on random sampling, and it selects the action with the highest expected reward.
• Deep reinforcement learning: Deep reinforcement learning is a combination of reinforcement learning and deep learning. It uses deep neural networks to represent the policy or value function, and it learns these representations by interacting with the environment. Deep reinforcement learning has been applied to a variety of complex tasks, including game playing and robot control.
• Actor-critic algorithms: Actor-critic algorithms are a class of reinforcement learning algorithms that use two separate networks: an actor network, which learns the policy, and a critic network, which learns the value function. The actor network is trained to maximize the reward, while the critic network is trained to evaluate the quality of the actions taken by the actor network.
• Policy gradient algorithms: Policy gradient algorithms are a class of reinforcement learning algorithms that directly optimize the policy function, which specifies the probability of taking each action in each state.

Reinforcement learning has many potential practical applications, including:

• Game playing: Reinforcement learning has been used to train agents to play a variety of games, such as chess, Go, and video games. By learning from the rewards and punishments received during gameplay, the agents can improve their performance and compete against human players.
• Robot control: Reinforcement learning has been used to train robots to perform a variety of tasks, such as navigating a maze, picking up objects, and playing catch. By learning from the rewards and punishments received during interaction with the environment, the robots can improve their performance and learn to perform complex tasks.
• Natural language processing: Reinforcement learning has been used to train agents to understand and generate natural language. By learning from the rewards received for generating coherent and grammatically correct sentences, the agents can improve their performance and generate human-like text.
• Finance: Reinforcement learning has been used to train agents to make trading decisions in financial markets. By learning from the rewards and punishments received for buying and selling stocks, the agents can improve their performance and make profitable trading decisions.
• Healthcare: Reinforcement learning has been used to train agents to make medical diagnosis and treatment decisions. By learning from the rewards and punishments received for making accurate diagnoses and effective treatments, the agents can improve their performance and provide better healthcare.

These are just a few examples of the many potential practical applications of reinforcement learning. As research in the field continues to advance, it is likely that new and exciting applications of reinforcement learning will be developed.

The Anatomy of Data

Data TYPES

• Numeric – a quantifiable number
  • Type – integer (e.g. age), floating (e.g. price), time, date, …
  • Stats – min/max/median/mean/…
  • Units – (C/F), (KG/Lb), (Meter/Feet), (Sec/Min/Hrs),
  • Distributions – exponential/uniform/…
• Ordinal – not quantifiable but ordered
  • E.g. size = Small/Medium/Large/…
  • E.g. income bucket = Low/Medium/High/Wealthy/…
  • E.g. Relevance = Perfect/Excellent/Good/Fair/Bad/…
• Symbolic – neither quantifiable nor ordered
  • E.g. color = red/green/blue/…
  • E.g. state/country/region/…
  • E.g. weather = rainy/cloudy/windy/…

Data MODALITIES:

• STRUCTURED – fixed columns in a table
  • Multivariate data
  • The mix of numeric and symbolic features
• UNSTRUCTURED — Arbitrary size data points
  • SEQUENCE: biological, speech, …
  • SERIES: stock market, etc.
  • TEXT: pages, queries, tweets, ads, blogs, news, …
  • IMAGE: regular, medical, remote sensing,…
  • VIDEO: regular, movies, security, surveillance,…

 

 

Learning a sequence.

You are given a sequence of numbers and asked to find the following number in the sequence.

0,1,1,2,3,5,8,13,21,34,55

Probably noticed that this is a Fibonacci sequence.

The first two terms are 0 and 1 and every term follows in the sum of its two predicting terms.

e.g

• 0 + 1 = 1
• 1 + 1 = 2
• 1+2=3
• 2 + 3 = 5
• 5 + 8 = 13
• 13 + 21=34
• 21 + 34 = 55

You can then keep predicting using the same model and generated a sequence long as you like.

The reason we come up with this answer is that we are on consciously trying to find a simple explanation for this data this is what we always do.

if further, I will ask you to expand this list you could enter that with your prediction.

The complexity of the model is defined using a hyperparameter.

“Learning also performs compression”

Pattern Recognition

Learning To read:

There are a lot of data are available which can be recognized by different visual formats.
E.g:

• QR code: scanned by smartphones
• Bar code: it is easy to read by the scanner, the code used in the product wrapper.

Optical character recognition is recognizing printed or written characters from their images.

We compare the seen input with all the prototypes one by one and choose the class with the best matching prototype this is called template matching

There may be errors in printing or sense, but we can do recognition by finding the closest match.

If we have many fonts for handwriting, we have multiple ways of writing the same character and we cannot possibly store all of them as possible templates. Instead, we want to learn the class by going over all the different examples of the same character and finding some general description that covers all of them.

Interestingly writing is a human invention so we collect samples from different writers, and fonts and learn the definition of “A”. All these distinct “A” have something in common that we want to extract.

We know that a character image is not just a collection of random dots and strokes of different orientations, but it has a regularity that we believe we can capture by using a learning program.

Matching model granularity

In machine learning, the aim is to feed a model to the data.

Parametric estimation: The model is trained with whole training data and all the instances of an effect, on the model parameters. In statistics, this is called parametric estimation.
In certain cases, we may have a set of local models each of which is applicable to a certain type of instance, this is a semiparametric estimation. The laptops that are most similar in their attributes it makes sense that their price should be similar too, is called the K nearest Neighbour algorithm where K is 3.

Generative model: An approach used to analyze and consider a generative model that represents our belief as to how the data is generated.

We know that size does not affect identity; this is called Invariance.

Face Recognition

• The input is the image captured by the camera and classes are the people to be recognized.
• The learning program should learn to match the face image to their identities.
• This problem is more difficult than optical character recognition because the input image is larger if the face is almost 3D and difference in poses and lighting causes significant changes in the image.
• The glasses may hide the eyes and eyebrows and the beard May hide the chin, this may affect the captured image.
• sometimes our faces change with respect to expressions of neutral happy angry this may affect the captured image.
• Affecting computing:  which aim is to have a computer system that can recognize and take into account human effects.

 Purpose of using Face Recognition:

• The aim is to identify the authentication of people.
• For full security purposes using face images is only one of the possibilities.
• Biometrics is the recognition or authentication of people using their physiological and behavioral characteristics.
• Along with face recognition, we can consider physiological characteristics like fingerprints, iris, and palms.
• Hopefully integrating multiple parameters will resolve this issue.

 

Speech Recognition

• The input Is the acoustic signal captured by the microphone and the classes are the words that can be uttered.
• We can consider each character image to be composed of basic primitives like strokes of different orientations, a word is made up of phonemes, which are the basic speech sounds.
• The speech may vary from person to person in age and gender Accent, and pronouncing the same word sounds differently.
• You can consider each word sound to be composed of two sets of factors 
  • Related to the word 
  • Related to speaker
• Speech recognition uses the first type of feature whereas speaker authentication uses the second.
• Unfortunately, the second type of feature is not easy to recognize or artificially generate, which is why the output of speech synthesis still sounds robotic.
• We can use a video image of the speaker’s Lips and the shape of the mouth.

Outlier Detection:

• The aim is to find instances that do not obey the general rule- those are the exceptions that are informative in certain contexts.
• An Outlier is an instance that is very much different from other instances in the sample.
• An Outlier may indicate an abnormal behavior of the system for example a credit card transaction it may indicate fraud. 
• An Outlier may also be recording errors due to a faulty sensor or any other factors.
• An Outlier may also be a novel, previously unseen but valid case, which is where the related term “Novelty detection” comes into play.

 

Overfitting and underfitting principles

 

Before proceeding further let’s go through the life of “Puspa”. 

Once Upon a Time, Pushpa asked his father “Papa what is a horse”. And father replied to Pushpa that, the animal that has four legs is called a horse.  Pushpa says papa your genius.

The next day, Pushpa went to school and saw a lot of animals with four legs, and she identified everything as a horse.

Unfortunately, Pushpa’s friend Anil was bitten by a dog, Anil cried loudly and ran to the teacher, Teacher asked “Anil what happened?  why your legs are full of blood” 

Anil was scared and could not speak anything, so the teacher asked Pushpa “What happened to Anil? Puspa replied that Anil was bitten by a horse. Teacher to Pushpa “what! But from where the horse came school campus. Are you telling me a lie Pushpa?”  then Pushpa replied “Teacher, God Promise, I have seen”, Then the teacher checked the CCTV and was aware of the dog bite.

The teacher called Pushpa and asked why she told lie to him, Pushpa answered about the definition of Horse taught by his father.

Now the teacher understands where the problem is. Then the teacher said to Pushpa “your father was right, not only four legs a horse, but in a horse, there are 2 long ears and a tail. In ancient times people were using horses for transportation.”

Pushpa said thank you, teacher!!!

The next day Pushpa asks the teacher “I have seen a horse near my house and a person was sitting on the horse and he was carrying some luggage through the horse, but the horse was so small and unable to run”

By holding a Donkey image in his hand, the teacher asked Pushpa “Have you seen a similar animal”

 Pushpa replied “yes teacher”

Now the teacher understood the problem and said to Pushpa “This is a donkey, and the donkey cannot run faster but a horse can run up to 55 mph and the donkey’s height is 35 inches 251 but the horse’s height is 55 inches 67 inch”

Now Pushpa thanks the teacher and knows the difference between a donkey and a horse.

During summer vacation Pushpa visited a Zoo with his father. Her father said to Pushpa “see this is mole”

Pushpa: “Papa this is a horse, why are you saying this is a mole?”  Then her father explained that a mole is a combination of horse and donkey and horse.

Now Pushpa knows the difference and similarities 

In Machine Learning, the same logic and intuition work here when we are designing a model. You can relate this model to Pushpa.

Underfitting:

• When Puspa recognized a dog as a horse due to very low intel as a horse has 4 legs, it is called Underfitted.
• The model is too basic and simple for the data.
• Data is quadratic and the model is linear. This situation is also called high bias. This means that the algorithm can do accurate predictions, but the initial assumption about the data is incorrect.

Overfitting:

• When Pushpa recognizes a mole as the horse is called Overfitted.
• The model is too complex for the data.
• The data is linear, and the model is a high-degree polynomial.
• This is also called high variance.

 

 

---

Below is an explanation Video.

---

 

Dimensionality Reduction

There are several different types of dimensionality reduction algorithms in machine learning. Some of the most common types include:

• Principal Component Analysis (PCA): This is a linear dimensionality reduction algorithm that projects the data onto a lower-dimensional space by selecting the most important features of the data.
• Singular Value Decomposition (SVD): This is a linear algebraic method for dimensionality reduction that decomposes the data matrix into three matrices, which can be used to identify the most important features of the data.
• Independent Component Analysis (ICA): This is a non-linear dimensionality reduction algorithm that attempts to find the underlying sources of the data by identifying the components that are statistically independent.
• t-Distributed Stochastic Neighbor Embedding (t-SNE): This is a non-linear dimensionality reduction algorithm that uses probability distributions to preserve the local structure of the data in the lower-dimensional space.
  Autoencoders: This is a type of neural network that is trained to encode the data into a lower-dimensional representation, and then decode that representation back into the original data. Autoencoders can be used for dimensionality reduction as well as for other tasks such as denoising and anomaly detection.

These are just a few examples of the many different types of dimensionality reduction algorithms that are used in machine learning. Each algorithm has its own strengths and weaknesses, and the appropriate algorithm for a given situation will depend on the specific characteristics of the data and the desired outcome.

Why Dimensionality Reduction?

• 1. Improved computational efficiency: Reducing the number of dimensions in the data can make it faster and more efficient to train machine learning models, especially for large and complex datasets. This is because training time and memory usage typically increase with the number of dimensions in the data.
  2. Enhanced data visualization: High-dimensional data can be difficult to visualize, as it is not possible to plot more than three dimensions on a 2D graph. Dimensionality reduction can transform the data into a lower-dimensional space, making it easier to visualize and understand the underlying patterns and trends.
  3. Reduced noise and redundancy: The high-dimensional space often contains a lot of noise and redundant information, which can interfere with the ability of machine learning algorithms to learn from the data. Dimensionality reduction can remove this noise and redundancy, resulting in cleaner and more relevant data for training.
  4. Improved performance of machine learning algorithms: Dimensionality reduction can improve the performance of machine learning algorithms by reducing overfitting and increasing the interpretability of the model. For example, by selecting the most important features of the data, dimensionality reduction can help to identify the underlying patterns and relationships that are most relevant for making predictions.
• Overall, dimensionality reduction is a valuable tool in the machine learning toolkit, as it can help to improve the efficiency and effectiveness of machine learning algorithms.

There are two ways to achieve dimensionality reduction:

• Feature selection and
• Feature extraction

Principal Component Analysis (PCA)

Principal component analysis (PCA) is a statistical technique that is used to analyze the variations in a data set. It is a dimensionality reduction technique that seeks to identify the underlying structure of the data in a way that reduces the complexity of the data set while still retaining as much of the variation in the data as possible.

• Standardize the data: PCA is sensitive to the scaling of the data, so it is important to first standardize the data by subtracting the mean and dividing by the standard deviation for each feature. This will ensure that all of the features are on the same scale and have a mean of zero.Standardizing the data in Principal Component Analysis (PCA) involves rescaling the variables in the dataset so that they have a mean of zero and a standard deviation of one. This is typically done by subtracting the mean of each variable from that variable and dividing by its standard deviation.Standardizing the data has several benefits. First, it ensures that each variable contributes equally to the analysis, since variables with larger variances will dominate the results if they are not standardized. Second, it makes it possible to compare the magnitude of the eigenvalues, which are a measure of the amount of variation in the data, since they will all be on the same scale. Finally, standardizing the data can sometimes improve the interpretability of the results, since the transformed variables will be in the same units as the original variables.To standardize the data in a dataset with n variables, you would first compute the mean and standard deviation of each variable, x_i, as follows:μ_i = mean(x_i)σ_i = std(x_i)Then, you would standardize each variable by subtracting its mean from that variable and dividing by its standard deviation:x_i’ = (x_i – μ_i) / σ_i
• Calculate the covariance matrix: The next step is to calculate the covariance matrix of the standardized data. The covariance matrix is a square matrix that contains the pairwise covariances between all of the features in the data.The covariance matrix is a square matrix that shows the covariance between different variables in a dataset. In the context of PCA, the covariance matrix is used to determine the directions (or “principal components”) that explain the maximum amount of variance in the data.To calculate the covariance matrix, you first need to center the data by subtracting the mean from each variable. Then, you can compute the covariance matrix as follows:Covariance matrix = (1 / (n – 1)) * X^T * Xwhere n is the number of observations in the dataset and X is the centered data matrix.Once you have the covariance matrix, you can then use it to determine the principal components of the data. This is typically done by finding the eigenvectors of the covariance matrix and ordering them by their corresponding eigenvalues (which indicate the amount of variance explained by each eigenvector). The eigenvectors with the largest eigenvalues are the principal components of the data.
• Calculate the eigenvectors and eigenvalues: The eigenvectors and eigenvalues of the covariance matrix are calculated next. The eigenvectors are the directions in the data that have the most variance, and the eigenvalues are the amount of variance along each eigenvector.To calculate the eigenvectors and eigenvalues of a matrix, you need to first compute the covariance matrix of the data. This can be done using the formula described in the previous answer: Covariance matrix = (1 / (n – 1)) * X^T * X, where n is the number of observations in the dataset and X is the centered data matrix. Once you have the covariance matrix, you can then use it to calculate the eigenvectors and eigenvalues. In most programming languages, there are built-in functions for doing this. For example, in Python, you can use the numpy.linalg.eig function to calculate the eigenvectors and eigenvalues of a matrix.The eigenvectors of the covariance matrix are the principal components of the data, and the corresponding eigenvalues indicate the amount of variance explained by each principal component. You can use the eigenvectors and eigenvalues to transform the original data into a new, lower-dimensional space that captures the most important variations in the data. This is the essence of PCA.
• Select the principal components: The number of dimensions in the resulting lower-dimensional space is determined by selecting the top k eigenvectors, where k is the desired number of dimensions. These eigenvectors are called the principal components, and they define the directions in the data along which the most variance is preserved.
• Project the data onto the principal components: The final step is to project the original data onto the principal components, resulting in a lower-dimensional representation of the data. This can be done by multiplying the standardized data by the matrix of eigenvectors, which will rotate and compress the data into the new lower-dimensional space.

Overall, the goal of PCA is to reduce the dimensionality of the data while preserving as much of the variance as possible. This can be useful for visualization, data compression, and noise reduction, among other applications.

 

Singular Value Decomposition (SVD)

Singular value decomposition (SVD) is a mathematical operation that decomposes a matrix into three separate matrices. It is commonly used in the fields of linear algebra and data analysis.

The SVD of a matrix X is written as X = U * S * V^T, where U and V are orthogonal matrices, and S is a diagonal matrix containing the singular values of X. The singular values are the square roots of the eigenvalues of X^T * X, and they indicate the importance of each of the dimensions in the original matrix.

SVD is often used in the context of principal component analysis (PCA), where it is used to determine the principal components of a dataset. It is also used in many other applications, including image and text analysis, recommendation systems, and data compression.

 

Independent component analysis (ICA)

Independent component analysis (ICA) is a statistical technique used to identify underlying sources of data that are mixed or correlated in some way. It is used to separate a multivariate signal into its independent components, which are assumed to be non-Gaussian and statistically independent from each other. This can be useful for a variety of applications, such as removing noise from a signal or identifying underlying patterns in the data.

 

t-Distributed Stochastic Neighbor Embedding (t-SNE)

t-Distributed Stochastic Neighbor Embedding (t-SNE) is a machine learning algorithm for visualizing high-dimensional data. It is commonly used for exploring and visualizing complex datasets, such as those encountered in natural language processing or genomics. The t-SNE algorithm maps the high-dimensional data points onto a lower-dimensional space, such that similar data points are mapped to nearby locations in the low-dimensional space. This allows for the creation of visualizations that can help reveal patterns and relationships in the data that might not be immediately apparent in the high-dimensional space.

Projection:

Mathematics is a fundamental part of machine learning, as it provides the tools and techniques needed to build, train, and evaluate machine learning models. Some of the key areas of mathematics that are used in machine learning include:

1. Linear algebra: Linear algebra is used to represent and manipulate data in machine learning. It is used to represent data points as vectors and matrices, and to perform operations such as matrix multiplication and vector dot products.
2. Calculus: Calculus is used to optimize machine learning models. It is used to find the minimum or maximum values of functions, which can be used to find the optimal values for the parameters of a machine learning model.
3. Probability and statistics: Probability and statistics are used to reason about and make predictions based on data in machine learning. It is used to calculate probabilities and to understand the relationships between different variables in a dataset.
4. Optimization: Optimization is used to find the best solution to a problem in machine learning. It is used to find the values of the parameters of a machine learning model that minimize the error or loss, and to find the best predictions given a set of data.

Overall, a strong foundation in mathematics is essential for anyone interested in pursuing a career in machine learning. It provides the tools and techniques needed to build, train, and evaluate machine learning models, and to solve real-world problems with data.

 

In this example, the linear regression model is implemented using the LinearRegression class from the sklearn.linear_model library. The model is trained on the given data, X and y, using the fit method, and then used to make predictions on new data using the predict method. The predicted values are then printed to the console.

Clustering

Clustering in machine learning is the process of dividing a dataset into groups, or clusters, based on the similarity of the data points within each cluster. This is an unsupervised learning technique, which means that the algorithm is not given any labels or categories to work with and must discover the underlying structure of the data on its own.

The goal of clustering is to group similar data points together and to identify patterns and trends in the data. Clustering algorithms can be used to classify data into different categories, to make predictions, and to improve the performance of other machine learning algorithms.

There are many different types of clustering algorithms, including k-means, hierarchical, and density-based clustering. These algorithms all work in different ways, but they all aim to identify clusters of similar data points within a dataset.

# import the necessary libraries
from sklearn.cluster import KMeans
import numpy as np

# create an array of data points
data = np.array([[1, 2], [3, 4], [5, 6], [7, 8], [9, 10]])

# create a k-means object with 2 clusters
kmeans = KMeans(n_clusters=2)

# fit the data and predict the clusters
clusters = kmeans.fit_predict(data)

# print the clusters
print(clusters)

This code creates an array of data points, and then uses the k-means algorithm to divide the data into two clusters. The clusters are then predicted and printed to the console.

Note that this is just a simple example to illustrate the basics of k-means clustering in Python. In a real-world application, you would need to use a larger dataset and you may need to fine-tune the parameters of the algorithm to get the best results.

Neural Network

A neural network is a type of machine learning algorithm that is modelled after the structure and function of the human brain. It is composed of multiple interconnected processing nodes, called neurons, which work together to solve complex problems.

 

Random forests

Gradient boosting

Gradient boosting is a machine learning algorithm that is used to create a model that sequentially adds weak learners (models that are only slightly better than random guessing) in order to boost the overall performance of the model. It is an ensemble learning method, which means that it combines multiple weak models to create a stronger, more accurate model.

The formula for gradient boosting is as follows:

1. Initialize the model with a constant value, such as the mean of the target variable.
2. For each iteration, train a weak learner to predict the residual error between the target variable and the current model prediction.
3. Update the model by adding the predicted residual error to the current model prediction.
4. Repeat steps 2 and 3 until the desired number of iterations is reached.

The practical use case for gradient boosting is to create highly accurate predictive models for complex, nonlinear problems. It is often used in applications such as customer churn prediction, fraud detection, and demand forecasting.

One of the key advantages of gradient boosting is that it can handle large datasets and high-dimensional data, and it can automatically learn nonlinear interactions between variables. This makes it a powerful tool for improving the performance of machine-learning models in a wide range of applications.

Naive Bayes

Naive Bayes is a classification algorithm that is used in machine learning to predict the probability of an outcome based on the prior probabilities of the features that make up the data. The algorithm is called “naive” because it makes the assumption that all the features in the data are independent of each other, which is not always the case in real-world data.

The formula for the Naive Bayes algorithm is as follows:

1. Calculate the prior probabilities for each class in the data, which is the probability that an instance belongs to that class.
2. For each feature in the data, calculate the likelihood, which is the probability that the feature has a given value given the class it belongs to.
3. Multiply the prior probabilities and the likelihoods together to get the posterior probability for each class, which is the probability that an instance belongs to a given class given the features it has.
4. Select the class with the highest posterior probability as the predicted class for the instance.

One practical use case for the Naive Bayes algorithm is spam filtering. In this application, the algorithm is trained on a dataset of email messages that are labeled as either “spam” or “not spam”. The features of the data could include the words in the email, the sender of the email, and other characteristics of the message. The algorithm then uses the calculated probabilities to predict whether a new email is a spam or not.

Another use case for the Naive Bayes algorithm is in medical diagnosis, where the algorithm can be trained on a dataset of patients with known diagnoses and symptoms. The features of the data could include the patient’s symptoms, medical test results, and other relevant information. The algorithm can then be used to predict the likelihood that a new patient has a particular disease based on their symptoms and test results.

---

How to create ML models that teach themselves