User Tools

Site Tools


extpgms
snippet.juliarepl
julia> pkgchk( [ "julia" => v"1.0.2" ] )

Executing External Programs (and Capturing Output)

Running and Capturing Single OS Programs

Julia can run arbitrary external programs, such as shell scripts, from within the REPL. The backtick notation is used to represent commands, although it is not used to run the command (unlike in many other scripting languages).

snippet.juliarepl
julia> mycommand= `echo "unix"`    ## do not run but define.  PS: backquoted allow interpolation
`echo unix`

julia> typeof( mycommand )        ## mycommand is *not* a string
Cmd

julia> run( mycommand )           ## now run, but do not capture
unix
Process(`echo unix`, ProcessExited(0))

julia> a= read( mycommand, String ) ## run and capture
"unix\n"

julia> a                          ## captured
"unix\n"

To write to unix commands:

snippet.juliarepl
julia> open(`grep -n ''`, "w", stdout ) do ofile;  for i=0:2; println(ofile, 'A'+i); end;   end#do
1:A
2:B
3:C

Pipelines

Julia does not pass the command string to the shell for interpretation. Instead, Julia rewrote shell functionality.

By intention, Julia does not allow passing entire shell sequences. For example, you cannot “run( ps | cut -c26-100 )”. (If you absolutely need the shell escape, you can always write the shell command to a shell script and then run the shell script.) Instead, Julia offers something better, which

  • reduces inadvertent shell-escape security holes, and
  • provides better error messages about which part of a pipe has failed in a pipeline (when it does).

Opening, Reading/Writing, and Closing Pipelines

Writing To a Pipe

snippet.juliarepl
julia>  open( pipeline(`gzip -c`, stdout="/tmp/pipelined.txt.gz"), "w" ) do opipe;
	    println(opipe, "this is gzip encoded,\nand here is our next line.\n")
	end#do##

FIXME Because this pipeline is really only one stage, this can also be written as open( gzip -c, "/tmp/pipelined.txt.gz", "w" ) ?!

Reading From a Pipe

snippet.juliarepl
julia>  open( pipeline(`gzcat /tmp/pipelined.txt.gz`), "r" ) do ipipe;
             readlines(ipipe)
	end#do##
3-element Array{String,1}:
"this is gzip encoded,"
"and here is our next line."
 ""

Reading from a Multi-State Pipe

snippet.juliarepl
julia> p= pipeline(`ps -ax` , `grep /sbin/launchd` , `grep ' 1 '` , `cut -c1–10,26100`)
pipeline(pipeline(pipeline(`ps -ax`, stdout=`grep /sbin/launchd`), stdout=`grep ' 1 '`), stdout=`cut -c1–10,26100`)

julia> run(p);
    1 ??  /sbin/launchd

PS: Note how the stdout= was automatically added to p. You could also provide destinations explicity, or even redirect stdin, stdout, and/or stderr.

See also Running External Commands.

FIXME mention that '&' can be used to simultaneously run multiple shell commands, too!

Accessing File and Process Components of The Pipeline

snippet.juliarepl
julia> pin= open( pipeline(`gzcat /tmp/pipelined.txt.gz`), "r" )    ## alternative reader with more detail info
Process(`gzcat /tmp/pipelined.txt.gz`, ProcessExited(0))

julia> readline( pin )       ## note: pin is Process <: IO object, so it can use IO functionality
"this is gzip encoded,"

julia> readline( pin )
"and here is our next line."

julia> readline( pin )
""

julia> readline( pin )       ## reading beyond the file end returns empty, and not an exception
""

julia> close( pin )
  • as always, readchomp() eliminates the final \n (but not intermediate ones).

Controlling Both Input and Output of External Programs or Sockets

Be careful…be very very careful. You may be waiting for program output to continue—but your program may be waiting for input from you.

Other Unix Programs

FIXME create an example that 'feeds to unix dc' and picks off the output.

feed into dc: 100 200 + f\n150 - f\n obtain:

snippet.juliafix
[download only julia statements]
writer = @async writeall(`cal`, "12 2000") ;;
reader = @async do_compute(reads(`sort, String`))
wait(process)
fetch(reader)

THIS NO LONGER WORKS

QUESTION FIXME Tell me how to do this.

snippet.julianoeval
[download only julia statements]
julia> (so,si,pr)= readandwrite(`cat`)
(Pipe(active, 0 bytes waiting),Pipe(open, 0 bytes waiting),Process(`cat`, ProcessRunning))
 
julia> write(si,"hello\ngood\nbye\n")
15
 
julia> so
Pipe(active, 15 bytes waiting)
 
julia> si
Pipe(open, 0 bytes waiting)
 
julia> close(si)
 
julia> so
Pipe(closed, 15 bytes waiting)
 
julia> pr
Process(`cat`, ProcessExited(0))
 
julia> readall(so)
"hello\ngood\nbye\n"
 
julia> so
Pipe(closed, 0 bytes waiting)

Network Socket

Adopted from Networking

FIXME the network socket docs were wrong in 0.6

listen, connect, getaddrinfo, etc.

listen(2001) == localhost:2000; listen(IPv4(0), 2001) is external; listen(“mysocket”) is a PipeServer.

snippet.juliafix
[download only julia statements]
julia> using Sockets
 
julia> @async begin
            server = listen(2000)
            while true
            while true
                sock = accept(server)
                @async while isopen(sock)
                   write(sock, readline(sock, keep=true))
                end
            end
        end##begin##
 Task (runnable) @0x00007fd31dc12e60
 
julia> clientside = connect(2001)
 TCPSocket(RawFD(28) open, 0 bytes waiting)
 
julia> @async while isopen(clientside)
           write(stdout, readline(clientside, keep=true))
        end;##while##
Task (runnable) @0x00007fd31dc11870

VERY OLD??

snippet.juliafix
[download only julia statements]
julia> mysocket= sock ;
       myserver=
           @async begin
           server = listen(2001)   ## == localhost:2000;
           while true
               global sock = accept(server)
               @async while isopen(sock)
                   rcvmsg= readline(sock)
                   (rcvmsg == "end\n") && break
                   write(sock,"received '$rcvmsg'\n")
               end
           end
       end##end##
Task (runnable) @0x00007fd31dc12e60
 
julia> clientside = connect(2001)
TCPSocket(RawFD(28) open, 0 bytes waiting)
 
julia> @async while true
           write(stdout,readline(clientside))
       end
Task (runnable) @0x00007fd31dc11870
 
julia> println(clientside,"Hello World from the Echo Server")
Hello World from the Echo Server
 
julia> close(clientside)

FIXME how to close the network socket server?

FIXME what does readavailable() do?

Backmatter

Useful Packages on Julia Repository

Notes

  • There is also mmap() that makes it possible to designate memory as an IOStream.
  • A wrapper that makes command calling simpler (but worse) is https://github.com/innerlee/Shell.jl .

References

extpgms.txt · Last modified: 2018/11/22 20:48 (external edit)