User Tools

Site Tools


system

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

system [2018/12/27 13:27] (current)
Line 1: Line 1:
 +
 +~~CLOSETOC~~
 +
 +~~TOC 1-3 wide~~
 +
 +```juliarepl
 +julia> pkgchk.( [ "​julia"​ => v"​1.0.3",​ "​Glob"​ => v"​1.2.0"​ ] );
 +
 +```
 +
 +
 +# Operating System
 +
 +## Identifying the Operating System
 +
 +```juliarepl
 +julia> homebase= Sys.isapple() ? "/​Users/"​ : Sys.islinux() ? "/​home/"​ : Sys.iswindows() ? "/"​ : nothing
 +"/​Users/"​
 +
 +```
 +
 +macOS is a BSD derivative, so `is_bsd()` would work, too.
 +
 +
 +## Identifying the Hostname
 +
 +```julianoeval
 +julia> gethostname()
 +"​pdp11.dec.com"​
 +```
 +
 +
 +## Finding the Name of Current Julia Program
 +
 +```julianoeval
 +julia> ;cat printinfo.jl
 +println(@__FILE__,​ ":",​ @__LINE__) ​        ## the current include(d) file or nothing
 +if (endswith(@__FILE__,​ PROGRAM_FILE))
 +    println("​you are the top level program or in the REPL")
 +end#if
 +
 +julia> include("​printinfo.jl"​) ## the full name, including path
 +/​Users/​me/​printinfo.jl:​1
 +you are the top level program
 +```
 +
 +* This can be abused for [[system#​unit_test_code|unit test code]] at the end of each individual .jl file.
 +
 +
 +## Command Line argument access (see programming)
 +
 +```juliarepl
 +julia> map( x->​string("​argument is ", x), ARGS)
 +0-element Array{String,​1}
 +```
 +
 +In the REPL, but not in a .jl file, you can also assign to ARGS.
 +
 +
 +For sophisticated option arguments to a program, julia features the [ArgParse](http://​argparsejl.readthedocs.io/​en/​latest/​argparse.html) package.
 +
 +
 +## Unix Environment
 +
 +Use `ENV`, which is an EnvHash. ​ For example,
 +
 +```juliarepl
 +julia> ENV[ "​SHELL"​ ]
 +"/​bin/​bash"​
 +```
 +
 +
 +## Exiting Julia
 +
 +```julianoeval
 +julia> exit(0) ​          ## by convention, 0 means clean exit.  positive means error
 +$
 +```
 +
 +
 +## Signals
 +
 +Please see [[parallel|Parallel Processing]].
 +
 +
 +
 +
 +# File System
 +
 +There are few examples in this section, because most functions should be self explanatory.
 +
 +
 +
 +## URLs
 +
 +^ **Function** ​ ^ **Explanation** ​ ^
 +| download(url[,​ localfile]) | download from URL, optionally renaming ​ |
 +
 +```julianoeval
 +julia> download("​http://​www.google.com",​ "/​tmp/​google.html"​)
 + % Total    % Received % Xferd  Average Speed   ​Time ​   Time     ​Time ​ Current
 +                                 ​Dload ​ Upload ​  ​Total ​  ​Spent ​   Left  Speed
 +100 12722    0 12722    0     ​0 ​  ​104k ​     0 --:--:-- --:--:-- --:​--:​-- ​ 105k
 +```
 +
 +
 +## File and Directory Operations
 +
 +### Copying, Moving, Removing, Symlinking
 +
 +^ **Function** ​ ^ **Explanation** ​ ^
 +| cp("​src",​ "​dst";​ remove_destination=false,​ follow_symlinks=false) | Copy file from src to dest.  |
 +| mv("​src",​ "​dst";​ remove_destination=false) | Move file from src to dst.  |
 +| rm("​path";​ recursive=false) | Delete file |
 +| touch("​path"​) ​ | Update the last-modified timestamp to now  |
 +| symlink(target,​ link)  | Create symbolic link to target ​ |
 +| readlink(path) ​ | Returns value of symbolic link path.  |
 +
 +
 +### Watching Files or Directory for Changes
 +
 +see below
 +
 +
 +
 +### Directories:​ Making, Connecting, Inquiring
 +
 +^ **Function** ​ ^ **Explanation** ​ ^
 +| pwd() | get working directory ​ |
 +| readdir([dir]) ​ | files and directories in the directory dir  |
 +| cd(["​dir"​=homedir()]) | set working directory. ​ |
 +| cd(f[, dir=homedir()]) | Temporarily change cwd, run f, and return. ​ See example. |
 +| mkdir(path[,​ mode]) | Make a new directory ​ |
 +| mkpath(path[,​ mode]) | Create all directories in the path  |
 +
 +
 +#### Examples
 +
 +```juliarepl
 +julia> mytmproot="/​tmp/​A/​B/​C/​D";​
 +
 +julia> mkpath(mytmproot);​ for i=1:5; touch("​$(mytmproot)/​fileis$(i+20)"​);​ end#for
 +
 +julia> readdir(mytmproot)
 +5-element Array{String,​1}:​
 + "​fileis21"​
 + "​fileis22"​
 + "​fileis23"​
 + "​fileis24"​
 + "​fileis25"​
 +
 +julia> filter( x -> occursin(r"​fileis.[34]",​ x), readdir(mytmproot) )        ## or use glob("​t?​[34]"​)
 +2-element Array{String,​1}:​
 + "​fileis23"​
 + "​fileis24"​
 +
 +julia> length( cd( readdir, mytmproot ) )
 +5
 +
 +julia> rm(mytmproot;​ recursive=true)
 +```
 +
 +
 +
 +## Temporary Files
 +
 +^ **Function** ​ ^ **Explanation** ​ ^
 +| tempname() | Generate a unique temporary file path. |
 +| tempdir() | Obtain the path of tempdir |
 +| mktemp([parent=tempdir()]) | Returns (path, io), where io is an open file object |
 +| mktemp(f::​Function[,​ parent=tempdir()]) | Apply function f to mktemp(parent) and then remove temp |
 +| mktempdir([parent=tempdir()]) | Create temporary directory |
 +| mktempdir(f::​Function[,​ parent=tempdir()]) | Apply function f to mktempdir(parent) and then remove temp |
 +
 +
 +#### Examples
 +
 +```julianoeval
 +julia> tempname()
 +"/​var/​folders/​yv/​xrwmg6zd2dx01b1rj5s7mzm00000z9/​T/​julia2RqITP"​
 +
 +julia> fout= mktemp()
 +("/​var/​folders/​yv/​xrwmg6zd2dx01b1rj5s7mzm00000z9/​T/​tmpAE1JQY",​ IOStream(<​fd 17>))
 +
 +julia> close(fout[2])
 +
 +julia> mktempdir( function(s::​String);​ println("​just working in ",s); end )
 +just working in /​var/​folders/​yv/​xrwmg6zd2dx01b1rj5s7mzm00000z9/​T/​tmpanQAp4
 +```
 +
 +Note that `Random.seed!()` has no control over the random filenames chosen.
 +
 +
 +
 +## Permissions,​ Types, and Inquiring about Files
 +
 +### Locking a File
 +
 +Julia has no built-in features for file locking.
 +
 +
 +### File Permissions and Timestamps
 +
 +^ **Function** ​ ^ **Explanation** ​ ^
 +| chmod(path, mode) | Change permission mode of path  |
 +| stat(file) | Return file information (e.g., size, dates) |
 +| ctime(file) | Equivalent to stat(file).ctime |
 +| mtime(file) | Equivalent to stat(file).mtime |
 +| filemode(file) | Equivalent to stat(file).mode |
 +| filesize(path...) | Equivalent to stat(file).size |
 +| uperm(file) | permissions of the owner of the file |
 +| gperm(file) | Like uperm but for group |
 +| operm(file) | Like uperm but for others |
 +
 +
 +#### Examples
 +
 +```julianoeval
 +julia> dump(stat("/​var/​tmp"​))
 +Base.Filesystem.StatStruct
 +  device: UInt64 0x0000000001000004
 +  inode: UInt64 0x000000000009cde6
 +  mode: UInt64 0x00000000000043ff
 +  nlink: Int64 4
 +  uid: UInt64 0x0000000000000000
 +  gid: UInt64 0x0000000000000000
 +  rdev: UInt64 0x0000000000000000
 +  size: Int64 128
 +  blksize: Int64 4194304
 +  blocks: Int64 0
 +  mtime: Float64 1.5342098751475687e9
 +  ctime: Float64 1.5342098751475687e9
 +
 +```
 +
 +
 +### Tests for Inode File Types
 +
 +^ **Function** ​ ^ **Explanation** ​ ^
 +| isblockdev(path) | true if path is a block device |
 +| ischardev(path) | true if path is a character device |
 +| isdir(path) | true if path is a directory |
 +| isexecutable(path) | true if permission to execute path |
 +| isfifo(path) | true if path is a FIFO |
 +| isfile(path) | true if path is a regular file |
 +| islink(path) | true if path is a symbolic link |
 +| ismount(path) | true if path is a mount point |
 +| ispath(path) | true if path is a valid filesystem path |
 +| isreadable(path) | true if permission to read path |
 +| issetgid(path) | true if path has the setgid flag set |
 +| issetuid(path) | true if path has the setuid flag set |
 +| issocket(path) | true if path is a socket |
 +| issticky(path) | true if path has the sticky bit set |
 +| iswritable(path) | true if permission to write to path |
 +
 +
 +### Testing for File Existence
 +
 +```juliarepl
 +julia> fileexists(path::​AbstractString)= ​ (stat(path).inode != 0)
 +fileexists (generic function with 1 method)
 +```
 +
 +
 +## Watching File For Changes (Non-Blocking I/O)
 +
 +See [File IO](https://​docs.julialang.org/​en/​stable/​stdlib/​io-network/​)
 +
 +```julianoeval
 +julia> using FileWatching
 +
 +julia> watch_file("/​var/​log/​system.log"​) ​         ## blocks until file change, reports old and new size
 +FileWatching.FileEvent(true,​ false, false)
 +
 +julia> poll_file("/​var/​log/​system.log"​) ​        ## blocks until file change, reports name
 +(StatStruct(mode=0o100664,​ size=16), StatStruct(mode=0o100664,​ size=17))
 +
 +```
 +
 +* These functions have timeout parameters.
 +
 +* There is no dot version that monitors multiple files. ​ However, there is a function that can watch folders, `watch_folder`.
 +
 +FIXME implement an async with a callback function...that is, the function is called everytime the file changes.
 +
 +
 +
 +## Path and Filename Parsing and Building
 +
 +^ **Function** ​ ^ **Explanation** ​ ^
 +| homedir() | Return user’s home directory. |
 +| dirname("​path"​) | Get the directory part. |
 +| basename("​path"​) | Get the file name part. |
 +| isabspath("​path"​) | Determines whether path begins at root directory |
 +| isdirpath("​path"​) | Determines whether path is a dir. |
 +| joinpath(parts...) | Join and clean path components into a full path. |
 +| abspath("​path"​) | Convert to an absolute path |
 +| normpath("​path"​) | Normalize, remove ”.” and ”..” . |
 +| realpath("​path"​) | Expand symbolic links and remove ”.” and ”..” . |
 +| relpath("​path",​ "​startpath"​ = "​."​) | Return a relative filepath |
 +| expanduser("​path"​) | On Unix, replace tilde with home directory |
 +| splitdir("​path"​) ​ | Split into a tuple of the directory name and file name. |
 +| splitdrive("​path"​) | On Windows, split into drive letter and path. |
 +| splitext("​path"​) | Split into file extension and else. |
 +
 +
 +
 +### Constructing Absolute Path From Current Directory
 +
 +```juliarepl
 +julia> using Glob;
 +
 +julia> map(abspath,​glob("​dataframe*.txt"​)) ​  ## construct absolute path
 +5-element Array{String,​1}:​
 + "/​Users/​ivo/​bitcookbook/​www/​farm/​julia/​data/​pages/​dataframecolumnops.txt"​
 + "/​Users/​ivo/​bitcookbook/​www/​farm/​julia/​data/​pages/​dataframecomplex.txt"​
 + "/​Users/​ivo/​bitcookbook/​www/​farm/​julia/​data/​pages/​dataframeintro.txt"​
 + "/​Users/​ivo/​bitcookbook/​www/​farm/​julia/​data/​pages/​dataframemissing.txt"​
 + "/​Users/​ivo/​bitcookbook/​www/​farm/​julia/​data/​pages/​dataframerowops.txt"​
 +
 +```
 +
 +
 +#### Examples
 +
 +```juliarepl
 +julia> function examinepath(p::​String = "​~/"​)
 +    for f in [ dirname, basename, isabspath, isdirpath, abspath, normpath, realpath, relpath, expanduser, splitdir, splitdrive, splitext ]
 +        println(f, ":​\t",​ f(p))
 +    end#for
 +end;#​function##​
 +
 +julia> examinepath("​."​)
 +dirname:
 +basename: .
 +isabspath:​ false
 +isdirpath:​ true
 +abspath:​ /​Users/​ivo/​bitcookbook/​www/​farm/​julia/​data/​pages/​
 +normpath: .
 +realpath:​ /​Users/​ivo/​bitcookbook/​www/​farm/​julia/​data/​pages
 +relpath: .
 +expanduser:​ .
 +splitdir:​ ("",​ "​."​)
 +splitdrive:​ ("",​ "​."​)
 +splitext:​ ("​.",​ ""​)
 +```
 +
 +`examinepath` also understands `expanduser(~)` and up-and-down traversals (e.g., `/​var/​../​etc`.
 +
 +
 +
 +
 +
 +
 +## Globbing and Processing Files
 +
 +### Glob and Reading File Directories (Folders)
 +
 +```juliarepl
 +julia> using Glob:glob
 +
 +julia> length(readdir("/​etc/​apache2/"​)) ​       ## readdir() [only] works on directory paths
 +9
 +
 +julia> length(glob("​*",​ "/​etc/​apache2/"​)) ​     ## glob does not allow first arg to start with /, use prefix in 2nd
 +9
 +
 +julia> length(glob("​*/",​ "/​etc/​apache2"​)) ​     ## subdirectories,​ because first arg ends with '/'​
 +4
 +
 +julia> length(glob("​http*",​ "/​etc/​apache2"​)) ​  ## glob pattern picks off specific files.
 +3
 +
 +julia> filter( x -> occursin(r"​.*/​httpd.c.*",​ x), glob("​*",​ "/​etc/​apache2"​)) ​  ## filter with regex is like glob
 +3-element Array{String,​1}:​
 + "/​etc/​apache2/​httpd.conf"​
 + "/​etc/​apache2/​httpd.conf.pre-update"​
 + "/​etc/​apache2/​httpd.conf~previous"​
 +```
 +
 +
 +
 +### Reading File Directories (Folders) Recursively
 +
 +from [https://​docs.julialang.org/​en/​stable/​stdlib/​file/​],​
 +
 +```julianoeval
 +julia> matchall(r::​Regex,​ s::​AbstractString;​ overlap::​Bool=false)= collect((m.match for m=eachmatch(r,​ s, overlap=overlap)))
 +matchall (generic function with 1 method)
 +
 +julia> function walklist( root::​String,​ MAXNUM::​Int=3 )
 +    i=0;
 +    rootlen= length(matchall(r"/",​ root))
 +    for (curroot, dirs, files) in walkdir(root;​ onerror=function(s);​ println("​**************** $(s) problem"​);​ end)
 +        i+=1
 +        curlen= length(matchall(r"/",​ curroot))
 +        println("​\n$(i) Working Dir $curroot: depth=",​ curlen - rootlen,";​ ndir(s)= $(length(dirs)),​ nfile(s)= $(length(files))"​)
 +        j= 0
 +        for dir in dirs
 +            j+=1; (j>​MAXNUM) && continue
 +            println(" ​ Subdir$j: ", joinpath(curroot,​ dir)) # path to directories
 +        end#for
 +        (j >= MAXNUM) && println("​\t...and $(j-3) other directories"​)
 +
 +        j= 0
 +        for file in files
 +            j+=1; (j>​MAXNUM) && continue
 +            println(" ​ File$j: ",​joinpath(curroot,​ file)) # path to files
 +        end#for
 +        (j>​MAXNUM) && println("​\t...and $(j-3) other files"​)
 +
 +        println("​$(i) Working Dir $curroot is done")
 +    end#for
 +end;#​function##​
 +
 +julia> walklist("/​etc/​apache2"​)
 +
 +1 Working Dir /​etc/​apache2:​ depth=0; ndir(s)= 4, nfile(s)= 5
 +  Subdir1: /​etc/​apache2/​extra
 +  Subdir2: /​etc/​apache2/​original
 +  Subdir3: /​etc/​apache2/​other
 + ...and 1 other directories
 +  File1: /​etc/​apache2/​httpd.conf
 +  File2: /​etc/​apache2/​httpd.conf.pre-update
 +  File3: /​etc/​apache2/​httpd.conf~previous
 + ...and 2 other files
 +1 Working Dir /​etc/​apache2 is done
 +
 +2 Working Dir /​etc/​apache2/​extra:​ depth=1; ndir(s)= 0, nfile(s)= 24
 +  File1: /​etc/​apache2/​extra/​httpd-autoindex.conf
 +  File2: /​etc/​apache2/​extra/​httpd-autoindex.conf~previous
 +  File3: /​etc/​apache2/​extra/​httpd-dav.conf
 + ...and 21 other files
 +2 Working Dir /​etc/​apache2/​extra is done
 +
 +3 Working Dir /​etc/​apache2/​original:​ depth=1; ndir(s)= 1, nfile(s)= 1
 +  Subdir1: /​etc/​apache2/​original/​extra
 +  File1: /​etc/​apache2/​original/​httpd.conf
 +3 Working Dir /​etc/​apache2/​original is done
 +
 +4 Working Dir /​etc/​apache2/​original/​extra:​ depth=2; ndir(s)= 0, nfile(s)= 12
 +  File1: /​etc/​apache2/​original/​extra/​httpd-autoindex.conf
 +  File2: /​etc/​apache2/​original/​extra/​httpd-dav.conf
 +  File3: /​etc/​apache2/​original/​extra/​httpd-default.conf
 + ...and 9 other files
 +4 Working Dir /​etc/​apache2/​original/​extra is done
 +
 +5 Working Dir /​etc/​apache2/​other:​ depth=1; ndir(s)= 0, nfile(s)= 1
 +  File1: /​etc/​apache2/​other/​php7.conf
 +5 Working Dir /​etc/​apache2/​other is done
 +
 +6 Working Dir /​etc/​apache2/​users:​ depth=1; ndir(s)= 0, nfile(s)= 2
 +  File1: /​etc/​apache2/​users/​Guest.conf
 +  File2: /​etc/​apache2/​users/​admin.conf
 +6 Working Dir /​etc/​apache2/​users is done
 +
 +```
 +
 +#### Asynch Version
 +
 +FIXME --- perhaps useful to do with @async and take!; ​ w=walkdir("/​etc"​);​ Channel{Any}(sz_max:​0,​sz_curr:​1)
 +
 +
 +
 +
 +
 +# Backmatter
 +
 +## Useful Packages on Julia Repository
 +
 +## Notes
 +
 +## References
 +
 +* See also [Julia Filesystem Documentation](https://​docs.julialang.org/​en/​stable/​stdlib/​file/​).
 +
 +* See also [Working with Text Files](https://​en.wikibooks.org/​wiki/​Introducing_Julia/​Working_with_text_files)
 +
 +* [Shelling Out Sucks](https://​julialang.org/​blog/​2012/​03/​shelling-out-sucks) and [Put this in Your Pipe](https://​julialang.org/​blog/​2013/​04/​put-this-in-your-pipe).
 +
 +* https://​en.wikibooks.org/​wiki/​Introducing_Julia/​Working_with_text_files#​Writing_to_files
  
system.txt · Last modified: 2018/12/27 13:27 (external edit)