User Tools

Site Tools


start

Julia Cookbook (0.7.0-alpha)

WARNING : As of June 13 2018, Julia 0.7.0 cannot yet be used, because many basic packages do not yet work. Use 0.6.3 for now.

About This Cookbook

  • The target audience for this cookbook are novice and intermediate Julia users with some prior programming experience (e.g., in R, perl, python, C, Fortran), and especially researchers with an interest in the social sciences.
  • This cookbook is for finding quick solutions of common patterns and questions, even for regular Julia programmers.
  • This cookbook is not well-suited to learning programming.
  • This cookbook is not well-suited for experienced julia programmers and developers, or those who want to read a book to become one.

The exposition methods are minimalistic examples, which should speak for themselves. The reader should immediately understand the functionality by looking at the (working) example. Long additional text explanations are (sometimes necessary) shortcomings.

About Julia

Julia is a next-generation programming language with many innovations. From my perspective, the most noteworthy general aspects about Julia are:

  • Julia arrays (vectors and matrices, easy to use for both Int and Float forms) are first-class design features, making Julia a great language for scientific computing.
  • Julia is strongly typed, making it easier both to find bugs earlier and to write efficient programs.
  • Julia relies greatly on function overloading, where a function with the same name can dispatch based on argument type.
  • Julia is compiled, with requisite compilation overhead. (Think of Julia more as C++, not as perl or R.) It is slow for short programs (compared to python and R), but it is (often blazingly) fast for long programs.

I am recommending Julia for programming, because:

  • Julia user code is cleaner and more logical than python or perl code, where (float) matrices are subsequent grafts. It is cleaner than R, which has never managed to decide whether it is primarily an interactive environment or a programming language. R's weak typing means bugs often manifest themselves very late, and debugging R can be akin to painful analysis of mystical incantations.
  • Julia is less bogged down by its own history than R, Python, or perl. The developers seem to get along and often help users (incl. early users like myself). The julia system has, as of 2018, not yet calcified to the point where even good and simple changes cause endless controversies. Every language must calcify sooner or later, but Julia will do so with the experiences of 2020, not with those of 1990.
  • Think of Julia as a “heavy-duty” tool and R as a “light-duty” tool. It would be silly to cut a hedge with scissors, or to cut paper with a hedge trimmer. Both can serve to do the other job in a pinch, but they really serve rather different purposes.

In my mind, Julia makes for a worse interactive environment than R (due to R's sloppy language and instant interpreted response speed), and a better programming environment than R (due to Julia's more careful and sane syntax and faster program run speed). Let me state this for the third time.

  • WARNING Julia is often an order of magnitude faster than R/python/perl for long programs. But Julia is also an order of magnitude slower than R/python/perl for short scripts and interactive use.

About JuliaComputing and JuliaPro

Julia is an open-source next-generation scientific computer language, designed by some amazing people at JuliaComputing (such as Jeff Bezanson, Alan Edelman, Stefan Karpinski, Viral B. Shah, and others). It has enthusiastic supporters that have answered many of my questions while I learned Julia (and while I wrote this cookbook). My own contribution for this cookbook was curating, i.e., keeping track of and organizing the smallest clear examples I could think of while I was learning Julia myself. The Perl Cookbook was my primary inspiration for this book. The perl cookbook is what made perl usable for me when I learned perl. (Thank you, Tom and Nathan!) I hope this Julia cookbook will help making Julia a success with new users.

JuliaComputing employs the core developers for Julia. To help finance Julia's development, please support them by purchasing their products:

  1. JuliaPro is Julia with a set of curated and more stable packages, an IDE, and enterprise support.
  2. JuliaBox is a web-based interface to a working and curated Julia, sold for a modest per month subscription price.

JuliaComputer further sells, but I have not yet used the following:

  1. JuliaBox is a web-based interface to a working and curated Julia, sold for a modest per month subscription price.
  2. JuliaRun is for large distributed cluster operations.
  3. JuliaDB is for operations on large disk-based data sets.

Julia Core Language

Julia consists of a core computer language and packages. Packages can add essential functionality (e.g., Missing values or DataFrame as dataset storage), programming tools (e.g., BenchmarkTools or Parallel Programming support), and/or entire applications (e.g., statistical analysis or plotting). Julia is not a lightweight language, with few principal features to remember, like C or lisp. It is a beast with many heads and appendages, tremendously powerful, but also easy to get lost in—similar to R (with CRAN), python (with PyPI), and perl (with CPAN) in this respect.

  • WARNING Some of the chapters below are nearly done. Others are just scrapbooks. The number of CHECKMARK indicates the progress. These checkmarks are updated irregularly.

0. Introduction

  1. Installation and Packages also REPL (Interactive) and Batch Use. CHECKMARK CHECKMARK
  2. Inquiring about object types and structures, methods and functions, naming conventions, etc. CHECKMARK CHECKMARK CHECKMARK

  3. Command Flow
  4. Scalars
  5. Strings
    1. Basics: concatenations, substrings, conversions, padding, capitalization, characterizations, parsing, escaping, etc. CHECKMARK CHECKMARK CHECKMARK
    2. Regular Expressions: finding and matching complex patterns. CHECKMARK CHECKMARK CHECKMARK

  6. Arrays and Tuples
  7. Associative Arrays (Key-Value)
    1. Dictionaries: (=maps=hashes), could be viewed as run-time structs. CHECKMARK CHECKMARK CHECKMARK
    2. Structures: could be viewed as compile-time dictionaries. CHECKMARK CHECKMARK CHECKMARK

  8. Programming and Functions
    1. Parallel Processing: Basics and Embarrassingly Parallel Processing CHECKMARK
    2. Programming Tools: Debugging, Profiling, Benchmarking, Unit-Testing, Writing Packages.

  9. DataFrames
    1. Introduction: building and accessing dataframes CHECKMARK CHECKMARK CHECKMARK
    2. Missing and NaN: in the context of data frames CHECKMARK CHECKMARK CHECKMARK
    3. Complex Operations: sorts, joins, by-group operations, wide-narrow conversions CHECKMARK CHECKMARK CHECKMARK
    4. Input/Output: csv files, binary IO, databases (SQL), and R data. FIXME when csv-IO stabilizes

More Julia Tools, Packages, Applications, and Notes

The next set of topics are primarily (but not exclusively) extended tools, packages, and applications rather than language topics.

  1. Files and Operating System
    1. Operating and File System: command-line args, OS detection, globbing, recursive directory traversals, etc. CHECKMARK CHECKMARK
    2. File Input/Output: opening, reading, writing, and closing files. CHECKMARK
    3. External Programs: running and capturing content from other programs. CHECKMARK
    4. JuliaDB: file-based statistical analysis on large disk-based data sets.

  2. Mathematical Search Algorithms
    1. JuMP Primarily for constrained Linear and Quadratic Problems. CHECKMARK CHECKMARK

  3. Statistics FIXME Tileman
    1. Univariate Statistics: tabulating, classifying, summary statistics, quantiles, CHECKMARK
  4. Plotting: Introduction Plots.jl FIXME Tileman
    1. Possibly not. With PyPlot
    2. Possibly not. With PlotlyJS
    3. Possibly not. With Gadfly — please avoid in 0.7. will improve.
    4. Possibly not. perhaps try StatPlots.jl (for integration with DataFrames, GR-based, by the same author as Plots.jl)

  5. Finance Support FIXME Tileman
    1. Prominent Free Finance Data Sources (Edgar, Fred, French, Intrinio, Etc.)
    2. Equities (Fama-Macbeth, Event Studies, Pfio Optimization, Factor-Analysis, Fama-French 92, etc.)
    3. Derivatives (Fixed Income, Options).
  6. Other Language Topics
    1. Translation Notes from R, Matlab, python, perl. FIXME

Background

Help, Authors, and License

  • If you want to help, please email me where I got things wrong.
  • This cookbook was written by Ivo Welch, with some early help from Pranav Bhat. Tileman Conring is coming on board, too.
  • Many of the solutions are based on help from folks at JuliaComputing and the julia discourse forum. Special thanks to Tamas_Papp, Nosferican, Mohamed82008, Steveng, StefanK, Elrod, and others.
  • This cookbook is taking intellectual credit only for the organization of the examples and exposition. We are merely curators. Julia and its packages were written by others, and it is these authors that deserve the intellectual credit. And ultimately, it is they who designed and thus they were really the only ones who could explain originally how to use their features. We hope to make their efforts more worthwhile by highlighting their work and showing how to use it.
  • This wiki uses DokuWiki, with the github-markdown syntax (and not the regular Dokuwiki syntax).
  • This book is free as in beer, but not free as in speech. It is copyrighted by its authors.

TODO

FIXME For the cookbook itself, write meta code:

  • style formatter
    • flag long lines
    • spaces in parenthenses should try to suggest nesting f(..g(.h(x).)..). Similiarly x=..
    • try to put all comments at column 72 with two ##, anding around column 100
  • LaTeX mapper
  • Jupyter mapper
  • Copy code button
start.txt · Last modified: 2018/06/14 01:58 by ivo