This shows you the differences between two versions of the page.
— |
system [2018/12/27 13:27] (current) |
||
---|---|---|---|
Line 1: | Line 1: | ||
+ | |||
+ | ~~CLOSETOC~~ | ||
+ | |||
+ | ~~TOC 1-3 wide~~ | ||
+ | |||
+ | ```juliarepl | ||
+ | julia> pkgchk.( [ "julia" => v"1.0.3", "Glob" => v"1.2.0" ] ); | ||
+ | |||
+ | ``` | ||
+ | |||
+ | |||
+ | # Operating System | ||
+ | |||
+ | ## Identifying the Operating System | ||
+ | |||
+ | ```juliarepl | ||
+ | julia> homebase= Sys.isapple() ? "/Users/" : Sys.islinux() ? "/home/" : Sys.iswindows() ? "/" : nothing | ||
+ | "/Users/" | ||
+ | |||
+ | ``` | ||
+ | |||
+ | macOS is a BSD derivative, so `is_bsd()` would work, too. | ||
+ | |||
+ | |||
+ | ## Identifying the Hostname | ||
+ | |||
+ | ```julianoeval | ||
+ | julia> gethostname() | ||
+ | "pdp11.dec.com" | ||
+ | ``` | ||
+ | |||
+ | |||
+ | ## Finding the Name of Current Julia Program | ||
+ | |||
+ | ```julianoeval | ||
+ | julia> ;cat printinfo.jl | ||
+ | println(@__FILE__, ":", @__LINE__) ## the current include(d) file or nothing | ||
+ | if (endswith(@__FILE__, PROGRAM_FILE)) | ||
+ | println("you are the top level program or in the REPL") | ||
+ | end#if | ||
+ | |||
+ | julia> include("printinfo.jl") ## the full name, including path | ||
+ | /Users/me/printinfo.jl:1 | ||
+ | you are the top level program | ||
+ | ``` | ||
+ | |||
+ | * This can be abused for [[system#unit_test_code|unit test code]] at the end of each individual .jl file. | ||
+ | |||
+ | |||
+ | ## Command Line argument access (see programming) | ||
+ | |||
+ | ```juliarepl | ||
+ | julia> map( x->string("argument is ", x), ARGS) | ||
+ | 0-element Array{String,1} | ||
+ | ``` | ||
+ | |||
+ | In the REPL, but not in a .jl file, you can also assign to ARGS. | ||
+ | |||
+ | |||
+ | For sophisticated option arguments to a program, julia features the [ArgParse](http://argparsejl.readthedocs.io/en/latest/argparse.html) package. | ||
+ | |||
+ | |||
+ | ## Unix Environment | ||
+ | |||
+ | Use `ENV`, which is an EnvHash. For example, | ||
+ | |||
+ | ```juliarepl | ||
+ | julia> ENV[ "SHELL" ] | ||
+ | "/bin/bash" | ||
+ | ``` | ||
+ | |||
+ | |||
+ | ## Exiting Julia | ||
+ | |||
+ | ```julianoeval | ||
+ | julia> exit(0) ## by convention, 0 means clean exit. positive means error | ||
+ | $ | ||
+ | ``` | ||
+ | |||
+ | |||
+ | ## Signals | ||
+ | |||
+ | Please see [[parallel|Parallel Processing]]. | ||
+ | |||
+ | |||
+ | |||
+ | |||
+ | # File System | ||
+ | |||
+ | There are few examples in this section, because most functions should be self explanatory. | ||
+ | |||
+ | |||
+ | |||
+ | ## URLs | ||
+ | |||
+ | ^ **Function** ^ **Explanation** ^ | ||
+ | | download(url[, localfile]) | download from URL, optionally renaming | | ||
+ | |||
+ | ```julianoeval | ||
+ | julia> download("http://www.google.com", "/tmp/google.html") | ||
+ | % Total % Received % Xferd Average Speed Time Time Time Current | ||
+ | Dload Upload Total Spent Left Speed | ||
+ | 100 12722 0 12722 0 0 104k 0 --:--:-- --:--:-- --:--:-- 105k | ||
+ | ``` | ||
+ | |||
+ | |||
+ | ## File and Directory Operations | ||
+ | |||
+ | ### Copying, Moving, Removing, Symlinking | ||
+ | |||
+ | ^ **Function** ^ **Explanation** ^ | ||
+ | | cp("src", "dst"; remove_destination=false, follow_symlinks=false) | Copy file from src to dest. | | ||
+ | | mv("src", "dst"; remove_destination=false) | Move file from src to dst. | | ||
+ | | rm("path"; recursive=false) | Delete file | | ||
+ | | touch("path") | Update the last-modified timestamp to now | | ||
+ | | symlink(target, link) | Create symbolic link to target | | ||
+ | | readlink(path) | Returns value of symbolic link path. | | ||
+ | |||
+ | |||
+ | ### Watching Files or Directory for Changes | ||
+ | |||
+ | see below | ||
+ | |||
+ | |||
+ | |||
+ | ### Directories: Making, Connecting, Inquiring | ||
+ | |||
+ | ^ **Function** ^ **Explanation** ^ | ||
+ | | pwd() | get working directory | | ||
+ | | readdir([dir]) | files and directories in the directory dir | | ||
+ | | cd(["dir"=homedir()]) | set working directory. | | ||
+ | | cd(f[, dir=homedir()]) | Temporarily change cwd, run f, and return. See example. | | ||
+ | | mkdir(path[, mode]) | Make a new directory | | ||
+ | | mkpath(path[, mode]) | Create all directories in the path | | ||
+ | |||
+ | |||
+ | #### Examples | ||
+ | |||
+ | ```juliarepl | ||
+ | julia> mytmproot="/tmp/A/B/C/D"; | ||
+ | |||
+ | julia> mkpath(mytmproot); for i=1:5; touch("$(mytmproot)/fileis$(i+20)"); end#for | ||
+ | |||
+ | julia> readdir(mytmproot) | ||
+ | 5-element Array{String,1}: | ||
+ | "fileis21" | ||
+ | "fileis22" | ||
+ | "fileis23" | ||
+ | "fileis24" | ||
+ | "fileis25" | ||
+ | |||
+ | julia> filter( x -> occursin(r"fileis.[34]", x), readdir(mytmproot) ) ## or use glob("t?[34]") | ||
+ | 2-element Array{String,1}: | ||
+ | "fileis23" | ||
+ | "fileis24" | ||
+ | |||
+ | julia> length( cd( readdir, mytmproot ) ) | ||
+ | 5 | ||
+ | |||
+ | julia> rm(mytmproot; recursive=true) | ||
+ | ``` | ||
+ | |||
+ | |||
+ | |||
+ | ## Temporary Files | ||
+ | |||
+ | ^ **Function** ^ **Explanation** ^ | ||
+ | | tempname() | Generate a unique temporary file path. | | ||
+ | | tempdir() | Obtain the path of tempdir | | ||
+ | | mktemp([parent=tempdir()]) | Returns (path, io), where io is an open file object | | ||
+ | | mktemp(f::Function[, parent=tempdir()]) | Apply function f to mktemp(parent) and then remove temp | | ||
+ | | mktempdir([parent=tempdir()]) | Create temporary directory | | ||
+ | | mktempdir(f::Function[, parent=tempdir()]) | Apply function f to mktempdir(parent) and then remove temp | | ||
+ | |||
+ | |||
+ | #### Examples | ||
+ | |||
+ | ```julianoeval | ||
+ | julia> tempname() | ||
+ | "/var/folders/yv/xrwmg6zd2dx01b1rj5s7mzm00000z9/T/julia2RqITP" | ||
+ | |||
+ | julia> fout= mktemp() | ||
+ | ("/var/folders/yv/xrwmg6zd2dx01b1rj5s7mzm00000z9/T/tmpAE1JQY", IOStream(<fd 17>)) | ||
+ | |||
+ | julia> close(fout[2]) | ||
+ | |||
+ | julia> mktempdir( function(s::String); println("just working in ",s); end ) | ||
+ | just working in /var/folders/yv/xrwmg6zd2dx01b1rj5s7mzm00000z9/T/tmpanQAp4 | ||
+ | ``` | ||
+ | |||
+ | Note that `Random.seed!()` has no control over the random filenames chosen. | ||
+ | |||
+ | |||
+ | |||
+ | ## Permissions, Types, and Inquiring about Files | ||
+ | |||
+ | ### Locking a File | ||
+ | |||
+ | Julia has no built-in features for file locking. | ||
+ | |||
+ | |||
+ | ### File Permissions and Timestamps | ||
+ | |||
+ | ^ **Function** ^ **Explanation** ^ | ||
+ | | chmod(path, mode) | Change permission mode of path | | ||
+ | | stat(file) | Return file information (e.g., size, dates) | | ||
+ | | ctime(file) | Equivalent to stat(file).ctime | | ||
+ | | mtime(file) | Equivalent to stat(file).mtime | | ||
+ | | filemode(file) | Equivalent to stat(file).mode | | ||
+ | | filesize(path...) | Equivalent to stat(file).size | | ||
+ | | uperm(file) | permissions of the owner of the file | | ||
+ | | gperm(file) | Like uperm but for group | | ||
+ | | operm(file) | Like uperm but for others | | ||
+ | |||
+ | |||
+ | #### Examples | ||
+ | |||
+ | ```julianoeval | ||
+ | julia> dump(stat("/var/tmp")) | ||
+ | Base.Filesystem.StatStruct | ||
+ | device: UInt64 0x0000000001000004 | ||
+ | inode: UInt64 0x000000000009cde6 | ||
+ | mode: UInt64 0x00000000000043ff | ||
+ | nlink: Int64 4 | ||
+ | uid: UInt64 0x0000000000000000 | ||
+ | gid: UInt64 0x0000000000000000 | ||
+ | rdev: UInt64 0x0000000000000000 | ||
+ | size: Int64 128 | ||
+ | blksize: Int64 4194304 | ||
+ | blocks: Int64 0 | ||
+ | mtime: Float64 1.5342098751475687e9 | ||
+ | ctime: Float64 1.5342098751475687e9 | ||
+ | |||
+ | ``` | ||
+ | |||
+ | |||
+ | ### Tests for Inode File Types | ||
+ | |||
+ | ^ **Function** ^ **Explanation** ^ | ||
+ | | isblockdev(path) | true if path is a block device | | ||
+ | | ischardev(path) | true if path is a character device | | ||
+ | | isdir(path) | true if path is a directory | | ||
+ | | isexecutable(path) | true if permission to execute path | | ||
+ | | isfifo(path) | true if path is a FIFO | | ||
+ | | isfile(path) | true if path is a regular file | | ||
+ | | islink(path) | true if path is a symbolic link | | ||
+ | | ismount(path) | true if path is a mount point | | ||
+ | | ispath(path) | true if path is a valid filesystem path | | ||
+ | | isreadable(path) | true if permission to read path | | ||
+ | | issetgid(path) | true if path has the setgid flag set | | ||
+ | | issetuid(path) | true if path has the setuid flag set | | ||
+ | | issocket(path) | true if path is a socket | | ||
+ | | issticky(path) | true if path has the sticky bit set | | ||
+ | | iswritable(path) | true if permission to write to path | | ||
+ | |||
+ | |||
+ | ### Testing for File Existence | ||
+ | |||
+ | ```juliarepl | ||
+ | julia> fileexists(path::AbstractString)= (stat(path).inode != 0) | ||
+ | fileexists (generic function with 1 method) | ||
+ | ``` | ||
+ | |||
+ | |||
+ | ## Watching File For Changes (Non-Blocking I/O) | ||
+ | |||
+ | See [File IO](https://docs.julialang.org/en/stable/stdlib/io-network/) | ||
+ | |||
+ | ```julianoeval | ||
+ | julia> using FileWatching | ||
+ | |||
+ | julia> watch_file("/var/log/system.log") ## blocks until file change, reports old and new size | ||
+ | FileWatching.FileEvent(true, false, false) | ||
+ | |||
+ | julia> poll_file("/var/log/system.log") ## blocks until file change, reports name | ||
+ | (StatStruct(mode=0o100664, size=16), StatStruct(mode=0o100664, size=17)) | ||
+ | |||
+ | ``` | ||
+ | |||
+ | * These functions have timeout parameters. | ||
+ | |||
+ | * There is no dot version that monitors multiple files. However, there is a function that can watch folders, `watch_folder`. | ||
+ | |||
+ | FIXME implement an async with a callback function...that is, the function is called everytime the file changes. | ||
+ | |||
+ | |||
+ | |||
+ | ## Path and Filename Parsing and Building | ||
+ | |||
+ | ^ **Function** ^ **Explanation** ^ | ||
+ | | homedir() | Return user’s home directory. | | ||
+ | | dirname("path") | Get the directory part. | | ||
+ | | basename("path") | Get the file name part. | | ||
+ | | isabspath("path") | Determines whether path begins at root directory | | ||
+ | | isdirpath("path") | Determines whether path is a dir. | | ||
+ | | joinpath(parts...) | Join and clean path components into a full path. | | ||
+ | | abspath("path") | Convert to an absolute path | | ||
+ | | normpath("path") | Normalize, remove ”.” and ”..” . | | ||
+ | | realpath("path") | Expand symbolic links and remove ”.” and ”..” . | | ||
+ | | relpath("path", "startpath" = ".") | Return a relative filepath | | ||
+ | | expanduser("path") | On Unix, replace tilde with home directory | | ||
+ | | splitdir("path") | Split into a tuple of the directory name and file name. | | ||
+ | | splitdrive("path") | On Windows, split into drive letter and path. | | ||
+ | | splitext("path") | Split into file extension and else. | | ||
+ | |||
+ | |||
+ | |||
+ | ### Constructing Absolute Path From Current Directory | ||
+ | |||
+ | ```juliarepl | ||
+ | julia> using Glob; | ||
+ | |||
+ | julia> map(abspath,glob("dataframe*.txt")) ## construct absolute path | ||
+ | 5-element Array{String,1}: | ||
+ | "/Users/ivo/bitcookbook/www/farm/julia/data/pages/dataframecolumnops.txt" | ||
+ | "/Users/ivo/bitcookbook/www/farm/julia/data/pages/dataframecomplex.txt" | ||
+ | "/Users/ivo/bitcookbook/www/farm/julia/data/pages/dataframeintro.txt" | ||
+ | "/Users/ivo/bitcookbook/www/farm/julia/data/pages/dataframemissing.txt" | ||
+ | "/Users/ivo/bitcookbook/www/farm/julia/data/pages/dataframerowops.txt" | ||
+ | |||
+ | ``` | ||
+ | |||
+ | |||
+ | #### Examples | ||
+ | |||
+ | ```juliarepl | ||
+ | julia> function examinepath(p::String = "~/") | ||
+ | for f in [ dirname, basename, isabspath, isdirpath, abspath, normpath, realpath, relpath, expanduser, splitdir, splitdrive, splitext ] | ||
+ | println(f, ":\t", f(p)) | ||
+ | end#for | ||
+ | end;#function## | ||
+ | |||
+ | julia> examinepath(".") | ||
+ | dirname: | ||
+ | basename: . | ||
+ | isabspath: false | ||
+ | isdirpath: true | ||
+ | abspath: /Users/ivo/bitcookbook/www/farm/julia/data/pages/ | ||
+ | normpath: . | ||
+ | realpath: /Users/ivo/bitcookbook/www/farm/julia/data/pages | ||
+ | relpath: . | ||
+ | expanduser: . | ||
+ | splitdir: ("", ".") | ||
+ | splitdrive: ("", ".") | ||
+ | splitext: (".", "") | ||
+ | ``` | ||
+ | |||
+ | `examinepath` also understands `expanduser(~)` and up-and-down traversals (e.g., `/var/../etc`. | ||
+ | |||
+ | |||
+ | |||
+ | |||
+ | |||
+ | |||
+ | ## Globbing and Processing Files | ||
+ | |||
+ | ### Glob and Reading File Directories (Folders) | ||
+ | |||
+ | ```juliarepl | ||
+ | julia> using Glob:glob | ||
+ | |||
+ | julia> length(readdir("/etc/apache2/")) ## readdir() [only] works on directory paths | ||
+ | 9 | ||
+ | |||
+ | julia> length(glob("*", "/etc/apache2/")) ## glob does not allow first arg to start with /, use prefix in 2nd | ||
+ | 9 | ||
+ | |||
+ | julia> length(glob("*/", "/etc/apache2")) ## subdirectories, because first arg ends with '/' | ||
+ | 4 | ||
+ | |||
+ | julia> length(glob("http*", "/etc/apache2")) ## glob pattern picks off specific files. | ||
+ | 3 | ||
+ | |||
+ | julia> filter( x -> occursin(r".*/httpd.c.*", x), glob("*", "/etc/apache2")) ## filter with regex is like glob | ||
+ | 3-element Array{String,1}: | ||
+ | "/etc/apache2/httpd.conf" | ||
+ | "/etc/apache2/httpd.conf.pre-update" | ||
+ | "/etc/apache2/httpd.conf~previous" | ||
+ | ``` | ||
+ | |||
+ | |||
+ | |||
+ | ### Reading File Directories (Folders) Recursively | ||
+ | |||
+ | from [https://docs.julialang.org/en/stable/stdlib/file/], | ||
+ | |||
+ | ```julianoeval | ||
+ | julia> matchall(r::Regex, s::AbstractString; overlap::Bool=false)= collect((m.match for m=eachmatch(r, s, overlap=overlap))) | ||
+ | matchall (generic function with 1 method) | ||
+ | |||
+ | julia> function walklist( root::String, MAXNUM::Int=3 ) | ||
+ | i=0; | ||
+ | rootlen= length(matchall(r"/", root)) | ||
+ | for (curroot, dirs, files) in walkdir(root; onerror=function(s); println("**************** $(s) problem"); end) | ||
+ | i+=1 | ||
+ | curlen= length(matchall(r"/", curroot)) | ||
+ | println("\n$(i) Working Dir $curroot: depth=", curlen - rootlen,"; ndir(s)= $(length(dirs)), nfile(s)= $(length(files))") | ||
+ | j= 0 | ||
+ | for dir in dirs | ||
+ | j+=1; (j>MAXNUM) && continue | ||
+ | println(" Subdir$j: ", joinpath(curroot, dir)) # path to directories | ||
+ | end#for | ||
+ | (j >= MAXNUM) && println("\t...and $(j-3) other directories") | ||
+ | |||
+ | j= 0 | ||
+ | for file in files | ||
+ | j+=1; (j>MAXNUM) && continue | ||
+ | println(" File$j: ",joinpath(curroot, file)) # path to files | ||
+ | end#for | ||
+ | (j>MAXNUM) && println("\t...and $(j-3) other files") | ||
+ | |||
+ | println("$(i) Working Dir $curroot is done") | ||
+ | end#for | ||
+ | end;#function## | ||
+ | |||
+ | julia> walklist("/etc/apache2") | ||
+ | |||
+ | 1 Working Dir /etc/apache2: depth=0; ndir(s)= 4, nfile(s)= 5 | ||
+ | Subdir1: /etc/apache2/extra | ||
+ | Subdir2: /etc/apache2/original | ||
+ | Subdir3: /etc/apache2/other | ||
+ | ...and 1 other directories | ||
+ | File1: /etc/apache2/httpd.conf | ||
+ | File2: /etc/apache2/httpd.conf.pre-update | ||
+ | File3: /etc/apache2/httpd.conf~previous | ||
+ | ...and 2 other files | ||
+ | 1 Working Dir /etc/apache2 is done | ||
+ | |||
+ | 2 Working Dir /etc/apache2/extra: depth=1; ndir(s)= 0, nfile(s)= 24 | ||
+ | File1: /etc/apache2/extra/httpd-autoindex.conf | ||
+ | File2: /etc/apache2/extra/httpd-autoindex.conf~previous | ||
+ | File3: /etc/apache2/extra/httpd-dav.conf | ||
+ | ...and 21 other files | ||
+ | 2 Working Dir /etc/apache2/extra is done | ||
+ | |||
+ | 3 Working Dir /etc/apache2/original: depth=1; ndir(s)= 1, nfile(s)= 1 | ||
+ | Subdir1: /etc/apache2/original/extra | ||
+ | File1: /etc/apache2/original/httpd.conf | ||
+ | 3 Working Dir /etc/apache2/original is done | ||
+ | |||
+ | 4 Working Dir /etc/apache2/original/extra: depth=2; ndir(s)= 0, nfile(s)= 12 | ||
+ | File1: /etc/apache2/original/extra/httpd-autoindex.conf | ||
+ | File2: /etc/apache2/original/extra/httpd-dav.conf | ||
+ | File3: /etc/apache2/original/extra/httpd-default.conf | ||
+ | ...and 9 other files | ||
+ | 4 Working Dir /etc/apache2/original/extra is done | ||
+ | |||
+ | 5 Working Dir /etc/apache2/other: depth=1; ndir(s)= 0, nfile(s)= 1 | ||
+ | File1: /etc/apache2/other/php7.conf | ||
+ | 5 Working Dir /etc/apache2/other is done | ||
+ | |||
+ | 6 Working Dir /etc/apache2/users: depth=1; ndir(s)= 0, nfile(s)= 2 | ||
+ | File1: /etc/apache2/users/Guest.conf | ||
+ | File2: /etc/apache2/users/admin.conf | ||
+ | 6 Working Dir /etc/apache2/users is done | ||
+ | |||
+ | ``` | ||
+ | |||
+ | #### Asynch Version | ||
+ | |||
+ | FIXME --- perhaps useful to do with @async and take!; w=walkdir("/etc"); Channel{Any}(sz_max:0,sz_curr:1) | ||
+ | |||
+ | |||
+ | |||
+ | |||
+ | |||
+ | # Backmatter | ||
+ | |||
+ | ## Useful Packages on Julia Repository | ||
+ | |||
+ | ## Notes | ||
+ | |||
+ | ## References | ||
+ | |||
+ | * See also [Julia Filesystem Documentation](https://docs.julialang.org/en/stable/stdlib/file/). | ||
+ | |||
+ | * See also [Working with Text Files](https://en.wikibooks.org/wiki/Introducing_Julia/Working_with_text_files) | ||
+ | |||
+ | * [Shelling Out Sucks](https://julialang.org/blog/2012/03/shelling-out-sucks) and [Put this in Your Pipe](https://julialang.org/blog/2013/04/put-this-in-your-pipe). | ||
+ | |||
+ | * https://en.wikibooks.org/wiki/Introducing_Julia/Working_with_text_files#Writing_to_files | ||