5 ms·

I tried out the dataset you mention. I use the latest Julia 1.5.2 and CSV.jl 0.7.7 and load the CSV package in 1s, read the file first time in 5s and then in 0.

I tried out the dataset you mention. I use the latest Julia 1.5.2 and CSV.jl 0.7.7 and load the CSV package in 1s, read the file first time in 5s and then in 0.03s the second time. The precompile time you mention is only for the very first time you download a new version of a package, which is then cached for subsequent sessions. Your generalization from a one time precompilation of the CSV.jl package to >100x slowdowns happening constantly is simply not true. julia> @time using CSV 1.161851 seconds (2.05 M allocations: 131.757 MiB, 0.46% gc time) julia> @time CSV.read("/Users/viral/Downloads/FL_insurance_sample.csv"); 4.884074 seconds (6.09 M allocations: 311.878 MiB, 1.90% gc time) julia> @time CSV.read("/Users/viral/Downloads/FL_insurance_sample.csv"); 0.030916 seconds (147.27 k allocations: 11.361 MiB)

orbifold 6y ago
So I want to plot the data I just loaded. Since I wrote the comment I had closed the Julia window (not to make a point, btw.) julia> import CSV (takes ~1-2 seconds) julia> data = CSV.File("/User/orbifold/Downloads/FL_insurance_sample.csv) (takes seconds again...) I want to explore the data. I vaguely remember there is Gadfly to do that. For some reason it depends on FFTW and a whole bunch of other packages, but the dependency story in Python is equally insane, so whatever. As I'm typing this, I'm waiting for Gadfly to precompile. julia> @time import Gadfly (104.460307 seconds) python> import matplotlib.pyplot as plt (~10s if you are unlucky and it has to generate the font cache) Maybe it's a good idea to do a scatter plot of two of the fields? Let's find out julia> @time Gadfly.plot(x=data.eq_site_limit, y=data.hu_site_limit) 0.320751 seconds but that number is not accurate, in fact it took so long for the browser to open the result, that I had time to check the documentation on backends and see whether I'm missing something. python> %time plt.scatter(data['eq_site_limit'], data['hu_site_limit']) 41.9 ms, a bit more for the window to open Turns out that is not a useful plot... maybe a histogram? julia> @time plot(x=data.eq_site_limit, Geom.histogram(bincount=10)) 0.042802 seconds It in fact again takes several seconds until the browser shows a plot. python> %time plt.hist(data["eq_site_limit"]) ~100ms and the window with the plot opens instantly.
- ViralBShah 6y ago
  Yes, Gadfly does have a large compile time. Reducing the time to first plot (essentially compilation time) has been a major focus for the 1.5 release. People often use Plots.jl or PyPlot which have significantly lesser compile times.
  - orbifold 6y ago
    time to precompile Plots: 138s julia> @time Plots.scatter(data.eq_site_limit, data.hu_site_limit) 5.461595 seconds subsequent calls report significant speed ups compared to matplotlib, but render about as quickly julia> @time Plots.histogram(data.eq_site_limit) 2.143983 seconds
    - ViralBShah 6y ago
      There are 3 different phases: 1. Precompilation time: Only once when you install or upgrade a package. 2. First running time (that includes compilation time): When you call a function like `plot` the first time in a session 3. Second and subsequent running times: The compiled code is now in the cache, and what you see is the actual running time. Thus what you are reporting here as precompilation (1) is not something that a user will see every time. What they will see is (2) for the first time in a session and (3) on subsequent plots. Could you confirm this is what you see - to ensure that those following along understand the terminology? Here are the usage times I see when I use Plots on a daily basis in a Julia session: ~ exec '/Users/viral/Desktop/Julia_Releases/Julia-1.5.app/Contents/Resources/julia/bin/julia' _ _ _ _(_)_ | Documentation: https://docs.julialang.org (_) | (_) (_) | _ _ _| |_ __ _ | Type "?" for help, "]?" for Pkg help. | | | | | | |/ _` | | | | |_| | | | (_| | | Version 1.5.2 (2020-09-23) _/ |\__'_|_|_|\__'_| | Official https://julialang.org/ release |__/ | julia> @time using Plots 11.341632 seconds (20.21 M allocations: 1.108 GiB, 2.87% gc time) julia> @time plot(1:10,1:10) 2.291147 seconds (3.24 M allocations: 165.569 MiB, 7.80% gc time) (Another 2 seconds for the window to be drawn the first time on mac) julia> @time plot(1:10,1:10) 0.000763 seconds (2.80 k allocations: 166.023 KiB)
  - 6y ago

orbifold 6y ago

So I want to plot the data I just loaded. Since I wrote the comment I had closed the Julia window (not to make a point, btw.) julia> import CSV (takes ~1-2 seconds) julia> data = CSV.File("/User/orbifold/Downloads/FL_insurance_sample.csv) (takes seconds again...) I want to explore the data. I vaguely remember there is Gadfly to do that. For some reason it depends on FFTW and a whole bunch of other packages, but the dependency story in Python is equally insane, so whatever. As I'm typing this, I'm waiting for Gadfly to precompile. julia> @time import Gadfly (104.460307 seconds) python> import matplotlib.pyplot as plt (~10s if you are unlucky and it has to generate the font cache) Maybe it's a good idea to do a scatter plot of two of the fields? Let's find out julia> @time Gadfly.plot(x=data.eq_site_limit, y=data.hu_site_limit) 0.320751 seconds but that number is not accurate, in fact it took so long for the browser to open the result, that I had time to check the documentation on backends and see whether I'm missing something. python> %time plt.scatter(data['eq_site_limit'], data['hu_site_limit']) 41.9 ms, a bit more for the window to open Turns out that is not a useful plot... maybe a histogram? julia> @time plot(x=data.eq_site_limit, Geom.histogram(bincount=10)) 0.042802 seconds It in fact again takes several seconds until the browser shows a plot. python> %time plt.hist(data["eq_site_limit"]) ~100ms and the window with the plot opens instantly.

ViralBShah 6y ago
Yes, Gadfly does have a large compile time. Reducing the time to first plot (essentially compilation time) has been a major focus for the 1.5 release. People often use Plots.jl or PyPlot which have significantly lesser compile times.
- orbifold 6y ago
  time to precompile Plots: 138s julia> @time Plots.scatter(data.eq_site_limit, data.hu_site_limit) 5.461595 seconds subsequent calls report significant speed ups compared to matplotlib, but render about as quickly julia> @time Plots.histogram(data.eq_site_limit) 2.143983 seconds
  - ViralBShah 6y ago
    There are 3 different phases: 1. Precompilation time: Only once when you install or upgrade a package. 2. First running time (that includes compilation time): When you call a function like `plot` the first time in a session 3. Second and subsequent running times: The compiled code is now in the cache, and what you see is the actual running time. Thus what you are reporting here as precompilation (1) is not something that a user will see every time. What they will see is (2) for the first time in a session and (3) on subsequent plots. Could you confirm this is what you see - to ensure that those following along understand the terminology? Here are the usage times I see when I use Plots on a daily basis in a Julia session: ~ exec '/Users/viral/Desktop/Julia_Releases/Julia-1.5.app/Contents/Resources/julia/bin/julia' _ _ _ _(_)_ | Documentation: https://docs.julialang.org (_) | (_) (_) | _ _ _| |_ __ _ | Type "?" for help, "]?" for Pkg help. | | | | | | |/ _` | | | | |_| | | | (_| | | Version 1.5.2 (2020-09-23) _/ |\__'_|_|_|\__'_| | Official https://julialang.org/ release |__/ | julia> @time using Plots 11.341632 seconds (20.21 M allocations: 1.108 GiB, 2.87% gc time) julia> @time plot(1:10,1:10) 2.291147 seconds (3.24 M allocations: 165.569 MiB, 7.80% gc time) (Another 2 seconds for the window to be drawn the first time on mac) julia> @time plot(1:10,1:10) 0.000763 seconds (2.80 k allocations: 166.023 KiB)
- 6y ago