Creation date: 2021-03-14
As part of an experiment we needed to solve a large quantity of linear inequalities described by the coefficient of a very sparse matrix.
I'm a novice of Julia but recently, on the Beginners AMA featuring Dr. Katharine Hyatt and Dr. Rachel Kurchin, I've heard about an interesting package called PyCall.jl. This provides the ability to call Python modules directly from Julia programs.
The setup was quite easy, I just had to follow the instructions of PyCall.jl's README about using a different version of Python as I have installed PyTorch in a Anaconda's environment.
It was really easy, I just copied the path to the correct executable and before doing
add PyCall I've set the
PYTHON environment variable directly from the Julia REPL then entered Pkg REPL-mode by pressing
] and finally added PyCall. For a full description of the procedure everything is described in the README mentioned above.
After that loading data from PyTorch was pretty straightforward but let me walk you through it:
using SparseArrays, PyCall @pyimport torch data_dict = torch.load("<path to my data>", map_location=torch.device("cpu")) C_rows = data_dict["ids"].numpy() .+ 1 C_cols = data_dict["ids"].numpy() .+ 1 C_values = data_dict["values"].numpy() C = sparse(C_rows, C_cols, C_values)
PyCall to import the
torch module from Python.
Called torch.load as if I was in Python (with the only difference that Julia doesn't allow for single quotes as string delimiters) to load my data, which is a dictionary of tensors.
By reading PyCall's README I've discovered that numpy's arrays are automatically converted into an interoperable array type in Julia. I just needed to add a +1 as Julia's indexing starts from 1 by default and I didn't really want to mess with custom indices.
After I've got all my data loaded I just needed to create my sparse matrix
sparse from the
SparseArrays module of the Standard Library.
So today we learned that Julia's interoperability with Python is quite easy, even with non standard modules like PyTorch, and that using Numpy's arrays to move data between languages is really neat and painless!