Before we take a look at the details of various machine learning methods, let’s start by looking at what machine learning is, and what it isn’t. Machine learning is often categorized as a subfield of artificial intelligence, but I find that categorization can often be misleading at first brush. The study of machine learning certainly arose from research in this context, but in the data science application of machine learning methods, it’s more helpful to think of machine learning as a means of *building models of data*.

Fundamentally, machine learning involves building mathematical models to help understand data. “Learning” enters the fray when we give these models tunable parameters that can be adapted to observed data; in this way the program can be considered to be “learning” from the data. Once these models have been fit to previously seen data, they can be used to predict and understand aspects of newly observed data. I’ll leave to the reader the more philosophical digression regarding the extent to which this type of mathematical, model-based “learning” is similar to the “learning” exhibited by the human brain.

Understanding the problem setting in machine learning is essential to using these tools effectively, and so we will start with some broad categorizations of the types of approaches we’ll discuss here.

OK 😎 un po’ più tranquillo 😁

**Categorie del machine learning**

At the most fundamental level, machine learning can be categorized into two main types: supervised learning and unsupervised learning.

- Supervised learning involves somehow modeling the relationship between measured features of data and some label associated with the data; once this model is determined, it can be used to apply labels to new, unknown data. This is further subdivided into classification tasks and regression tasks: in classification, the labels are discrete categories, while in regression, the labels are continuous quantities. We will see examples of both types of supervised learning in the following section.
- Unsupervised learning involves modeling the features of a dataset without reference to any label, and is often described as “letting the dataset speak for itself.” These models include tasks such as clustering and dimensionality reduction. Clustering algorithms identify distinct groups of data, while dimensionality reduction algorithms search for more succinct representations of the data. We will see examples of both types of unsupervised learning in the following section.

In addition, there are so-called semi-supervised learning methods, which falls somewhere between supervised learning and unsupervised learning. Semi-supervised learning methods are often useful when only incomplete labels are available.

**Esempi qualitativi di applicazioni machine learning**

To make these ideas more concrete, let’s take a look at a few very simple examples of a machine learning task. These examples are meant to give an intuitive, non-quantitative overview of the types of machine learning tasks we will be looking at in this chapter. In later sections, we will go into more depth regarding the particular models and how they are used. For a preview of these more technical aspects, you can find the Python source that generates the following figures in the Appendix: Figure Code.

**classificazione: predire etichette discrete**

We will first take a look at a simple classification task, in which you are given a set of labeled points and want to use these to classify some unlabeled points.

Per compltezza copio anche il codice disponibile all’URL indicato. Notare che manca l’istruizione ** import matplotlib.pyplot as plt**.

`from sklearn.datasets.samples_generator import make_blobs from sklearn.svm import SVC # create 50 separable points X, y = make_blobs(n_samples=50, centers=2, random_state=0, cluster_std=0.60) # fit the support vector classifier model clf = SVC(kernel='linear') clf.fit(X, y) # create some new points to predict X2, _ = make_blobs(n_samples=80, centers=2, random_state=0, cluster_std=0.80) X2 = X2[50:] # predict the labels y2 = clf.predict(X2) # plot the data import matplotlib.pyplot as plt fig, ax = plt.subplots(figsize=(8, 6)) # common plot formatting for below def format_plot(ax, title): ax.xaxis.set_major_formatter(plt.NullFormatter()) ax.yaxis.set_major_formatter(plt.NullFormatter()) ax.set_xlabel('feature 1', color='gray') ax.set_ylabel('feature 2', color='gray') ax.set_title(title, color='gray') format_plot(ax, 'Input Data') point_style = dict(cmap='Paired', s=50) ax.scatter(X[:, 0], X[:, 1], c=y, **point_style) ax.axis([-1, 4, -2, 7]) fig.savefig('no868.png')`

Here we have two-dimensional data: that is, we have two features for each point, represented by the (x,y) positions of the points on the plane. In addition, we have one of two class labels for each point, here represented by the colors of the points. From these features and labels, we would like to create a model that will let us decide whether a new point should be labeled “blue” or “red.”

There are a number of possible models for such a classification task, but here we will use an extremely simple one. We will make the assumption that the two groups can be separated by drawing a straight line through the plane between them, such that points on each side of the line fall in the same group. Here the model is a quantitative version of the statement “a straight line separates the classes”, while the model parameters are the particular numbers describing the location and orientation of that line for our data. The optimal values for these model parameters are learned from the data (this is the “learning” in machine learning), which is often called training the model.

The following figure shows a visual representation of what the trained model looks like for this data:

`from sklearn.datasets.samples_generator import make_blobs from sklearn.svm import SVC import numpy as np import matplotlib.pyplot as plt # common plot formatting for below def format_plot(ax, title): ax.xaxis.set_major_formatter(plt.NullFormatter()) ax.yaxis.set_major_formatter(plt.NullFormatter()) ax.set_xlabel('feature 1', color='gray') ax.set_ylabel('feature 2', color='gray') ax.set_title(title, color='gray') # create 50 separable points X, y = make_blobs(n_samples=50, centers=2, random_state=0, cluster_std=0.60) # fit the support vector classifier model clf = SVC(kernel='linear') clf.fit(X, y) # Get contours describing the model xx = np.linspace(-1, 4, 10) yy = np.linspace(-2, 7, 10) xy1, xy2 = np.meshgrid(xx, yy) Z = np.array([clf.decision_function([t]) for t in zip(xy1.flat, xy2.flat)]).reshape(xy1.shape) # plot points and model fig, ax = plt.subplots(figsize=(8, 6)) point_style = dict(cmap='Paired', s=50) line_style = dict(levels = [-1.0, 0.0, 1.0], linestyles = ['dashed', 'solid', 'dashed'], colors = 'gray', linewidths=1) ax.scatter(X[:, 0], X[:, 1], c=y, **point_style) ax.contour(xy1, xy2, Z, **line_style) # format plot format_plot(ax, 'Model Learned from Input Data') ax.axis([-1, 4, -2, 7]) fig.savefig('np869.png')`

Now that this model has been trained, it can be generalized to new, unlabeled data. In other words, we can take a new set of data, draw this model line through it, and assign labels to the new points based on this model. This stage is usually called prediction. See the following figure:

`from sklearn.datasets.samples_generator import make_blobs from sklearn.svm import SVC import numpy as np import matplotlib.pyplot as plt # common plot formatting for below def format_plot(ax, title): ax.xaxis.set_major_formatter(plt.NullFormatter()) ax.yaxis.set_major_formatter(plt.NullFormatter()) ax.set_xlabel('feature 1', color='gray') ax.set_ylabel('feature 2', color='gray') ax.set_title(title, color='gray') # create 50 separable points X, y = make_blobs(n_samples=50, centers=2, random_state=0, cluster_std=0.60) # fit the support vector classifier model clf = SVC(kernel='linear') clf.fit(X, y) # Get contours describing the model xx = np.linspace(-1, 4, 10) yy = np.linspace(-2, 7, 10) xy1, xy2 = np.meshgrid(xx, yy) Z = np.array([clf.decision_function([t]) for t in zip(xy1.flat, xy2.flat)]).reshape(xy1.shape) # create 50 separable points X, y = make_blobs(n_samples=50, centers=2, random_state=0, cluster_std=0.60) # fit the support vector classifier model clf = SVC(kernel='linear') clf.fit(X, y) # create some new points to predict X2, _ = make_blobs(n_samples=80, centers=2, random_state=0, cluster_std=0.80) X2 = X2[50:] # predict the labels y2 = clf.predict(X2) point_style = dict(cmap='Paired', s=50) line_style = dict(levels = [-1.0, 0.0, 1.0], linestyles = ['dashed', 'solid', 'dashed'], colors = 'gray', linewidths=1) # plot the results fig, ax = plt.subplots(1, 2, figsize=(16, 6)) fig.subplots_adjust(left=0.0625, right=0.95, wspace=0.1) ax[0].scatter(X2[:, 0], X2[:, 1], c='gray', **point_style) ax[0].axis([-1, 4, -2, 7]) ax[1].scatter(X2[:, 0], X2[:, 1], c=y2, **point_style) ax[1].contour(xy1, xy2, Z, **line_style) ax[1].axis([-1, 4, -2, 7]) format_plot(ax[0], 'Unknown Data') format_plot(ax[1], 'Predicted Labels') fig.savefig('np870.png')`

This is the basic idea of a classification task in machine learning, where “classification” indicates that the data has discrete class labels. At first glance this may look fairly trivial: it would be relatively easy to simply look at this data and draw such a discriminatory line to accomplish this classification. A benefit of the machine learning approach, however, is that it can generalize to much larger datasets in many more dimensions.

For example, this is similar to the task of automated spam detection for email; in this case, we might use the following features and labels:

- feature 1, feature 2, etc. → normalized counts of important words or phrases (“Viagra”, “Nigerian prince”, etc.)
- label → “spam” or “not spam”

For the training set, these labels might be determined by individual inspection of a small representative sample of emails; for the remaining emails, the label would be determined using the model. For a suitably trained classification algorithm with enough well-constructed features (typically thousands or millions of words or phrases), this type of approach can be very effective. We will see an example of such text-based classification in In Depth: Naive Bayes Classification [prossimamente].

Some important classification algorithms that we will discuss in more detail are Gaussian naive Bayes (see In Depth: Naive Bayes Classification [prossimamente]), support vector machines (see In-Depth: Support Vector Machines [prossimamente]), and random forest classification (see In-Depth: Decision Trees and Random Forests [prossimamente]).

Pausa 😯 poi si continua 😊