Why is this Elixir code so slow? -
i'm interested in learning elixir , typical approach of learning new language build simple programs it.
so decided write (very simple) grep-like program (or module) shown here:
defmodule linematch def file(s, path) case file.open(path, [:read]) {:ok, fd} -> match s, fd {_, error} -> io.puts "#{:file.format_error error}" end end defp match(s, fd) case io.read fd, :line {:error, error} -> io.puts("oh noez! error: #{error}") line -> match(s, line, fd) end end defp match(s, line, fd) when line !== :eof if string.contains?(line, s) io.write line end match(s, io.read(fd, :line), fd) end defp match(_, _line, _) when _line === :eof end end
this not elegant way , i'm quite new functional programming, didn't expect super fast. not not fast, super slow. slow did super obvious mistake.
can tell me, , how make better?
i typically test code seperate .exs file like
case system.argv [searchterm, path] -> linematch.file(searchterm, path) _ -> io.puts "usage: linematch searchterm path" end
rather reading in whole file in lad2025's answer, can performance doing 2 things. first, use io.binstream
build stream of file's lines, raw binary (for performance). using io.stream
reads utf-8, , such incurs cost conversion reading file. if need utf-8 conversion, it's going slow. additionally, applying filtering , mapping operations using stream.filter/2
, stream.map/2
functions prevents iterating on lines multiple times.
defmodule grepper def grep(path, str) case file.open(path) {:error, reason} -> io.puts "error grepping #{path}: #{reason}" {:ok, file} -> io.binstream(file, :line) |> stream.filter(&(string.contains?(&1, str))) |> stream.map(&(io.puts(io.ansi.green <> &1 <> io.ansi.reset))) |> stream.run end end end
i suspect greatest issue code utf-8 conversion, may "pulling" file line line calling io.read
, rather having lines "pushed" filtering/printing operations using io.stream|binstream
, incurring cost there. i'd have @ elixir's source know sure, above code quite performant on machine (i searching 143kb file olson timezone database, not sure how perform on files of large size don't have sample file handy).
Comments
Post a Comment