I’m new to Julia and Flux so I suspect this may be something dumb that I may be missing…
I was trying to do some preprocessing in order to train a NN using Flux and ended up stuck with two problems:

The predictions of the Neural Network were all
NaN
. I suspected that the parameters were not being updated but it turns out that they were but still I get onlyNaN
s. 
The
Standardizer()
transformer from MLJ is not able to applyinverse_transform
to the normalized arrays for the evaluation of the NN as it results in aLoadError: type Nothing has no field names
(see stacktrace below). The weird thing is that the transformation seems to have been carried out as expected when the values of the normalized DataFrames are printed.
I also had problems with the Standardizer()
not scaling Int64
features in the DataFrame which were resolved by just converting manually (lines 21:23). I wonder if there’s any better way to do it.
The data was downloaded from this Kaggle dataset page
The whole script that reproduces these two problems is pasted below:
using Flux
using CSV
using DataFrames
using MLJ
using ScientificTypes
using Tables
function preprocess(x, y; train_size = 0.8)
columns_to_delete = [
"FLUVENTS",
"DYSTROPEPTS",
"ORTHENTS",
"UDALFS",
"USTALFS",
]
select!(x, Not(columns_to_delete))
for feature in names(x)
x[!, feature] = convert.(Float64, x[:, feature])
end
limit = trunc(Int64, size(x, 1) * train_size)
xtrain = x[begin:limit, :]
xtest = x[limit:end, :]
ytrain = y[begin:limit, :]
ytest = y[limit:end, :]
xtrain_std_mach = machine(Standardizer(), xtrain)
ytrain_std_mach = machine(Standardizer(), ytrain)
xtest_std_mach = machine(Standardizer(), xtest)
ytest_std_mach = machine(Standardizer(), ytest)
norm_xtrain = MLJ.transform(fit!(xtrain_std_mach), xtrain)
norm_xtest = MLJ.transform(fit!(xtest_std_mach), xtest)
norm_ytrain = MLJ.transform(fit!(ytrain_std_mach), ytrain)
norm_ytest = MLJ.transform(fit!(ytest_std_mach), ytest)
norm_xtrain = Array(norm_xtrain)'
norm_xtest = Array(norm_xtest)'
norm_ytrain = Array(norm_ytrain)'
norm_ytest = Array(norm_ytest)'
return norm_xtrain, norm_ytrain, norm_xtest, norm_ytest, ytest_std_mach
end
function train(xtrain, ytrain; epochs::Int64)
dataloader = Flux.DataLoader((xtrain, ytrain), batchsize = 4, shuffle = true)
model = Chain(
Dense(16, 100, relu),
Dense(100, 100, relu),
Dense(100, 1, sigmoid),
)
loss(x, y) = Flux.Losses.mse(model(x), y)
optimizer = Flux.ADAM()
for current_epoch in range(1, epochs)
println("Epoch $current_epoch/$epochs")
Flux.train!(loss, Flux.params(model), dataloader, optimizer)
end
return model
end
function evaluate(model, xtest, ytest, target_scaler)
norm_preds = model(xtest) # all preds are NaN
real = inverse_transform(target_scaler, ytest) # can't inverse transform data
pred = inverse_transform(target_scaler, norm_preds)
println(Flux.Losses.mse(preds, real))
end
function main()
x = CSV.read("./X1.csv", DataFrame)
y = CSV.read("./y1.csv", DataFrame)
xtrain, ytrain, xtest, ytest, target_scaler = preprocess(x, y)
model = train(xtrain, ytrain, epochs = 15)
evaluate(model, xtest, ytest, target_scaler)
return
end
main()
The corresponding stack trace:
[ Info: Training Machine{Standardizer,…}.
[ Info: Training Machine{Standardizer,…}.
┌ Warning: Extremely small standard deviation encountered in standardization.
└ @ MLJModels ~/.julia/packages/MLJModels/4sRmw/src/builtins/Transformers.jl:500
┌ Warning: Extremely small standard deviation encountered in standardization.
└ @ MLJModels ~/.julia/packages/MLJModels/4sRmw/src/builtins/Transformers.jl:500
┌ Warning: Extremely small standard deviation encountered in standardization.
└ @ MLJModels ~/.julia/packages/MLJModels/4sRmw/src/builtins/Transformers.jl:500
┌ Warning: Extremely small standard deviation encountered in standardization.
└ @ MLJModels ~/.julia/packages/MLJModels/4sRmw/src/builtins/Transformers.jl:500
[ Info: Training Machine{Standardizer,…}.
[ Info: Training Machine{Standardizer,…}.
ERROR: LoadError: type Nothing has no field names
Stacktrace:
[1] getproperty
@ ./Base.jl:42 [inlined]
[2] _standardize
@ ~/.julia/packages/MLJModels/4sRmw/src/builtins/Transformers.jl:876 [inlined]
[3] inverse_transform(#unused#::Standardizer, fitresult::NamedTuple{(:is_univariate, :is_invertible, :fitresult_given_feature), Tuple{Bool, Bool, Dict{Symbol, Tuple{Float64, Float64}}}}, X::LinearAlgebra.Adjoint{Float64, Matrix{Float64}})
@ MLJModels ~/.julia/packages/MLJModels/4sRmw/src/builtins/Transformers.jl:861
[4] inverse_transform(mach::Machine{Standardizer, true}, Xraw::LinearAlgebra.Adjoint{Float64, Matrix{Float64}})
@ MLJBase ~/.julia/packages/MLJBase/u6vLz/src/operations.jl:88
[5] evaluate(model::Chain{Tuple{Dense{typeof(relu), Matrix{Float32}, Vector{Float32}}, Dense{typeof(relu), Matrix{Float32}, Vector{Float32}}, Dense{typeof(σ), Matrix{Float32}, Vector{Float32}}}}, xtest::LinearAlgebra.Adjoint{Float64, Matrix{Float64}}, ytest::LinearAlgebra.Adjoint{Float64, Matrix{Float64}}, target_scaler::Machine{Standardizer, true})
@ Main ~/Documents/projetos/minicursoML/texte/model.jl:84
[6] main()
@ Main ~/Documents/projetos/minicursoML/texte/model.jl:99
[7] toplevel scope
@ ~/Documents/projetos/minicursoML/texte/model.jl:104
[8] include
@ ./client.jl:451 [inlined]
[9] toplevel scope
@ ./timing.jl:210 [inlined]
[10] toplevel scope
@ ./REPL[11]:0
[11] toplevel scope
@ ~/.julia/packages/CUDA/YpW0k/src/initialization.jl:52
in expression starting at /var/home/enzo/Documents/projetos/minicursoML/texte/model.jl:104
Some info about the package versions:
julia> versioninfo()
Julia Version 1.7.0beta4.2
Commit d0c90f37ba (20210824 12:35 UTC)
Platform Info:
OS: Linux (x86_64redhatlinux)
CPU: Intel(R) Core(TM) i78550U CPU @ 1.80GHz
WORD_SIZE: 64
LIBM: libopenlibm
LLVM: libLLVM12.0.1 (ORCJIT, skylake)
(@v1.7) pkg> status
Status `~/.julia/environments/v1.7/Project.toml`
[27a7e980] Animations v0.4.1
[6e4b80f9] BenchmarkTools v1.2.0
[336ed68f] CSV v0.9.10
[5ae59095] Colors v0.12.8
[a93c6f00] DataFrames v1.2.2
[864edb3b] DataStructures v0.18.10
[b4f34e82] Distances v0.10.6
[587475ba] Flux v0.12.8
[78b212ba] Javis v0.7.1
[add582a8] MLJ v0.16.11
[91a5bcdd] Plots v1.23.4
[321657f4] ScientificTypes v2.3.3
[bd369af6] Tables v1.6.0
Any thoughts on what I may be missing or suggestions on how this workflow could be improved?
Thanks a lot in advance!!