As we’ve already seen in previous sections, the power of the PyData stack is built upon the ability of NumPy and Pandas to push basic operations into C via an intuitive syntax: examples are vectorized/broadcasted operations in NumPy, and grouping-type operations in Pandas. While these abstractions are efficient and effective for many common use cases, they often rely on the creation of temporary intermediate objects, which can cause undue overhead in computational time and memory use.

As of version 0.13 (released January 2014), Pandas includes some experimental tools that allow you to directly access C-speed operations without costly allocation of intermediate arrays. These are the ** eval()** and

**functions, which rely on the Numexpr package. In this notebook we will walk through their use and give some rules-of-thumb about when you might think about using them.**

`query()`

**Perché query() e eval(): espressioni composte**

We’ve seen previously that NumPy and Pandas support fast vectorized operations; for example, when adding the elements of two arrays:

As discussed in Computation on NumPy Arrays: Universal Functions [qui], this is much faster than doing the addition via a Python loop or comprehension:

But this abstraction can become less efficient when computing compound expressions. For example, consider the following expression:

`mask = (x > 0.5) & (y < 0.5)`

Because NumPy evaluates each subexpression, this is roughly equivalent to the following:

`tmp1 = (x > 0.5)`

`tmp2 = (y < 0.5)`

`mask = tmp1 & tmp2`

In other words, every intermediate step is explicitly allocated in memory. If the ** x** and

**arrays are very large, this can lead to significant memory and computational overhead. The Numexpr library gives you the ability to compute this type of compound expression element by element, without the need to allocate full intermediate arrays. The Numexpr documentation has more details, but for the time being it is sufficient to say that the library accepts a string giving the NumPy-style expression you’d like to compute:**

`y`

The benefit here is that Numexpr evaluates the expression in a way that does not use full-sized temporary arrays, and thus can be much more efficient than NumPy, especially for large arrays. The Pandas ** eval()** and

**tools that we will discuss here are conceptually similar, and depend on the Numexpr package.**

`query()`

`pandas.eval()`

**per operazioni efficienti**

The ** eval()** function in Pandas uses string expressions to efficiently compute operations using

**s. For example, consider the following**

`DataFrame`

**s:**

`DataFrame`

To compute the sum of all four ** DataFrame**s using the typical Pandas approach, we can just write the sum:

The same result can be computed via ** pd.eval** by constructing the expression as a string:

The ** eval()** version of this expression is about 50% faster (and uses much less memory), while giving the same result:

**Operazioni supportate da** `pd.eval()`

As of Pandas v0.16, ** pd.eval()** supports a wide range of operations. To demonstrate these, we’ll use the following integer

**s:**

`DataFrame`

**operatori aritmetici**

** pd.eval()** supports all arithmetic operators. For example:

**operatori di comparazione**

** pd.eval()** supports all comparison operators, including chained expressions:

**operatori bitwise**

** pd.eval()** supports the

**and**

`&`

**bitwise operators:**

`|`

In addition, it supports the use of the literal and and or in Boolean expressions:

**attributi di oggetti e indici**

** pd.eval()** supports access to object attributes via the

**syntax, and indexes via the**

`obj.attr`

**syntax:**

`obj[index]`

**altre operazioni**

Other operations such as function calls, conditional statements, loops, and other more involved constructs are currently not implemented in ** pd.eval()**. If you’d like to execute these more complicated types of expressions, you can use the Numexpr library itself.