Rust e Python – 1

b0

Davide Aversa, TheK3nger ha postato la trilogia How to use Rust in Python, qui part 1, part 2 e part 3.

Ora:

  • Python lo conosco e consiglio;
  • Rust lo sto studiando;
  • TheK3nger rockz 😀

e allora rileggendo i post citati faccio gli esercizi proposti 😀
Quando ci arriverò nel mio studio di Rust qualcosa di simile è trattato nella guida in Foreign Function Interface [/usr/local/share/doc/rust/html/book/ffi.html] ma usa il C come altro linguaggio. Per me Python è più usato e allora benvenuto a questi post k3ngerosi 😉

Python è lento, Rust è molto promettente e allora, Rust is the perfect language for embedding fast-binary libraries in Python!

Anche perché [w]riting Rust code that can be executed in Python is stupidly easy.

Partiamo con Rust e Cargo:

k0

In lib.rs inseriamo il seguente codice:

use std::thread;

fn process() {
    let handles: Vec = (0..10).map(|_| {
        thread::spawn(|| {
            let mut x = 0;
            for _ in 0..5_000_000 {
                x += 1
            }
            x
        })
    }).collect();

    for h in handles {
        println!("Thread finished with count={}",
        h.join().map_err(|_| "Could not join a thread!").unwrap());
    }
}

Nota: in realtà c’è nel Git il file finale, completo e commentato, probabilmente si userà quello.
In order to make this piece of code working as an external library we need to change the function definition with the following lines:

#[no_mangle]
pub extern fn process() {
...

Don’t worry about the meaning of this for now. Just note that extern means that we want to call this function from the outside, from a C-like interface, and  pub means that the function is “public”.

The second change we need is to add these two lines at the end of the Cargo.toml file.

[lib]
name = "rustbomb"
crate-type = ["dylib"]

This say to the compiler to compile the library in a “standard” way so that can be used as a C library, and not in the “rusty” way. We can now compile this crate and we are ready to embed this library in a Python script.

Provo a compilare ma…

k1

provo a sostituire i files lib.rs e Cargo.toml con quelli del Git:

k2

OK! 😀

Scrivere il lato Python

We need now to write the Python part of the application. Create a new file client.py (for instance) in the crate root [cioè rustbomb]. To attach the Python script to the library we will use the standard interface ctypes.

import ctypes

# Wind DLL get as parameter the path to the brand new compiled library!
# corretto per Linux
lib = ctypes.CDLL("./target/release/librustypython.so")

lib.process()

print("done")

We finished in just 3 lines of code. Try to run this Python script and enjoy some rusty threads running on your cores!

k3

😀 OK 😀 Thek3nger rockz! (forse già detto ma repetita Juventus (& Moggi birikinassaye)).

Passare parametri alle funzioni Rust

What if we want to pass as parameter the number of threads we want to spawn?
The first naive implementation will be to just add the parameters in Rust in the standard way.

#[no_mangle]
pub extern fn process2(threads_number: i32) {
let handles: Vec<_> = (0..threads_number).map(|_| {
thread::spawn(|| {
...

And then call this new function with a number. And… Well.. It works! However, is not a wise to use “Rust’s specific data types” when we are writing an FFI interface. We need something more C-friendly.

Aggiungiamo a Cargo.toml:

[dependencies]
libc = "0.2.0"

This will include the libc crate that will add a lot of C-friendly types you can use. Then we need to add these two lines at the beginning of the Rust library.

extern crate libc;

use libc::{size_t,int32_t};

Now we are ready to write the function declaration with the right type:

#[no_mangle]
pub extern fn process2(threads_number: int32_t) {
let handles: Vec<_> = (0..threads_number).map(|_| {
thread::spawn(|| {
...

Note that we are using int32_t instead of i32. It is the same thing, but now your code is bulletproof, even if you change architecture or implementations. It is formally correct and it safer to use.

Passo a part 2

[W]e have seen how to run simple Rust functions with integer arguments. This is not enough, of course. We need to go further by passing Python lists to Rust functions.

The problem is that it is not possible to pass directly a Python list to a C interface. Python lists (we can call them Plists) are complicated beasts, you can easily see that they are objects full of methods, and attributes and… Stuff.

k4

We need first to convert this in something edible from a Rust library. But first things first.

La funzione Rust somma di una lista di interi

The first step to do is to write the Rust function in our library. This time we want a function that, given a list of integers as input, returns the sum of the list. The final code is something like this:

#[no_mangle]
pub extern fn process2(threads_number: int32_t) {
    let handles: Vec = (0..threads_number).map(|_| {
        thread::spawn(|| {
            let mut x = 0;
            for _ in 0..5_000_000 {
                x += 1
            }
            x
        })
    }).collect();

    for h in handles {
        println!("Thread finished with count={}",
        h.join().map_err(|_| "Could not join a thread!").unwrap());
    }
}

It is not a complicated function, but you can see that there are some interesting points:

The function takes two arguments: a pointer to an int32_t which represent the first item on our list; and a size_t value representing the size of the list. We cannot pass a list directly, but we need to pass the list in its “primordial form”: an array of data in memory and the length of that array.
Because the list is given as raw data, we need to “assemble” the list according the Rust fashion. This is what I’m doing in the unsafe block: create a slice from the raw parts.

Then the function continues as usual, folding the list in order to get the final sum. (Remember that no semi-colon at the end of a statement is just like a “return” cioè l’ultima espressione valutata è il valore della funzione, proprio come in Lisp; se metto il punto-virgola l’ultima espressione vale null.

Il lato Python

Ecco la funzione sum_list, per me il file sl.py:

import ctypes

lib = ctypes.CDLL("./target/release/librustypython.so")
lib.sum_list.argtypes = (ctypes.POINTER(ctypes.c_int32), ctypes.c_size_t)
print("Summing in Rust the list of first 1000 numbers.")
number_list = list(range(1001))
c_number_list = (ctypes.c_int32 * len(number_list))(*number_list)
result = lib.sum_list(c_number_list, len(number_list))
print("Result is {}. Expected 500500.".format(result))

k5

Cosa succede:

  • First line. This explicitly defines the type of the arguments of sum_list. To be honest the script works even without this line, but I think it is better to use it anyway.
  • c_number_list is where we explode the pythonic list into a C friendly list. It is a bit cryptic, but in short this mean: create a new Python object representing an array of len(number_list) blocks of c_int32; then initialize this object with the contents  of the desired list (number_list).
  • Then we simply call the Rust function with this object (representing the raw data) and the length (given by the length of the list).

A nice side effect. Because we are using just the raw content of the list by “unpacking” the list (the asterisk before number_list on line 4), the same code works even if number_list is a tuple or a set.

Già detto che TheK3nger rockz? 😀 E non finisce qui, c’è in arrivo una seconda puntata.

:mrgreen:

Posta un commento o usa questo indirizzo per il trackback.

Trackback

  • Rust e Python – 2 | Ok, panico su 11 marzo 2016 alle 08:47

    […] da qui l’esame della serie di post di TheK3nger sull’uso di Rust all’interno di codice […]

  • Linux e Windows (e me) | Ok, panico su 24 marzo 2016 alle 17:01

    […] dev’essere limitato. E (rigarda me) Rust è troppo difficile (ho anche copiato TheK3nger, qui e qui ma non ho convinto). E i linguaggi funzionali (Lisp e tutti gli altri) no, solo per me. Ma in […]

  • Rust – il linguaggio – 51 | Ok, panico su 20 aprile 2016 alle 08:21

    […] è già stato trattato in modo comprensibile anche agli umani come me, copiando TheKeng3r, qui e […]

Rispondi

Inserisci i tuoi dati qui sotto o clicca su un'icona per effettuare l'accesso:

Logo di WordPress.com

Stai commentando usando il tuo account WordPress.com. Chiudi sessione /  Modifica )

Google photo

Stai commentando usando il tuo account Google. Chiudi sessione /  Modifica )

Foto Twitter

Stai commentando usando il tuo account Twitter. Chiudi sessione /  Modifica )

Foto di Facebook

Stai commentando usando il tuo account Facebook. Chiudi sessione /  Modifica )

Connessione a %s...

Questo sito utilizza Akismet per ridurre lo spam. Scopri come vengono elaborati i dati derivati dai commenti.

%d blogger hanno fatto clic su Mi Piace per questo: