Skip to content
Hallway
0:00 / 0:00

#emacs#org-mode#reproducibleresearch

Emacs + org-mode + python in reproducible research; SciPy 2013 Presentation

Enthought

68K subscribers • 4 videos on Hallway

YouTube
  • Recommended by 5 people: 5 on Hacker News.
  • A classic. People kept recommending it from 2014 to 2018.
  • Presented by John Kitchin, Professor at Carnegie Mellon University

Intermediate. Assumes working knowledge.

49K views13 years ago5 recommendations

Tools and practiceProgramming languagesVim, Emacs and editorsPython

John Kitchin demonstrates how to use Emacs org-mode and Python to create reproducible research documents that integrate code, data, and narrative.

Authors: Kitchin, John Carnegie Mellon University Track: Reproducible Science We will discuss the use of emacs + org-mode + python in enabling reproducible research. This combination of software enables researchers to intertwine narrative and mathematical text with figures and code that is executable within a document, with capture of the output. Portions of the document can be selectively exported to LaTeX, HTML, pdf and other other formats. We have used this method to produce technical manuscripts submitted for peer review in scientific journals, in the preparation of two e-books (about 300 pages each) on using python in scientific and engineering applications (jkitchin.github.com/pycse), and in using python in the modeling of the properties of materials with density functional theory (jkitchin.github.com/dft-book), as well as a python-powered blog at jkitchin.github.com. Our experience suggests all three components are critical for enabling reproducible research in practice: an extensible editor, a markup language that separates text, math, data and code, and an effective language such as python. We will show examples of the pros and cons of this particular implementation of editor/markup/code combination.

Why it's here

5 people recommend this

What people say. Recommended as a classic demonstration of using Emacs org-mode for literate programming and reproducible scientific research.

cube_yellowon Hacker News • Recommends

On “Let’s Make a Bubble Map”

A lot of people are starting to use org-mode for that kind of thing. Fully reproducible research in a single file. To check the work, just export the file to pdf/html/etc., or "C-c C-c" the code blocks as you read through in org-mode. It's fantastic. For python specifically, this video is a must watch: link to this video
View on Hacker News

hsitzon Hacker News • Recommends

On “IPython: Beginners’ Guide”

The Orgmode extension to Emacs has quite an elegant, flexible, and powerful way of doing what you want. Can be hard to wrap your mind around at the beginning but has good payoff once you're up to speed. Here's a video showing how one guy uses it with Python (though it integrates with multitude of languages, including R): link to this video Here's paper (itself from LaTeX generated by Orgmode) describing use of Orgmode for this kind of "reproducible research": link
View on Hacker News

taericon Hacker News • Recommends

On “Patoline: A modern digital typesetting system”

I know it gets said a lot, but have you considered emacs and org-mode. Because, it goes a very long way for this. See this great talk showing its use with python. link to this video
View on Hacker News

taericon Hacker News • Recommends

On “Show HN: Lit – a modern literate programming tool”

Do check out org-mode, then. It is surprising how far that tool has been extended. Many of these features are actually under the label "reproducible research" nowdays. There was a really good talk given on this at a pypy convention a couple of years ago.1 1 link to this video
View on Hacker News

taericon Hacker News • Recommends

On “Typographic Programming Language”

Seriously, try emacs. You want to see what color you have put in a css field?1 Want to see the image you are referencing in a repl?2 More amazingly, want to combine TeX, lisp, a markdown like language, and python/whatever in a single document? Try org-mode.34 1 link 2 link 3 link 4 link to this video
View on Hacker News

rndnon Hacker News • Recommends

On “Emacs for Data Science”

This SciPy presentation (Kitchin, John Carnegie Mellon University) on Emacs + org-mode + python in reproducible research is worth a watch: link to this video
View on Hacker News

daharton Hacker News • Recommends

On “Jupyter, Mathematica, and the Future of the Research Paper”

I Agree with all those points. I've used org mode for a few things, but not for python notebooks. I remember seeing a great video a few years back on using python in org mode to make a research notebook. It really made me want to try org mode for literate / notebook style programming. I think it was this one: link to this video It sounds like org mode does not store images inline, since you don't like the b64 images in Jupyter? I imagine that could go both ways depending on what you want. Even for diffing, you might like to see that the image changed (though I generally agree that diffing images is not ideal). If you have to package images separately when sharing, then wouldn't inline images score a po…
View on Hacker News

u/Bogdanpin r/compsci • Mentions

On “My current way of storing research results is way too chaotic. Is there a tool that helps me get organized?”

Org-mode for Emacs might be what you're looking for: link to this video
8View on Reddit

cube_yellowon Hacker News • Mentions

On “Anki – Powerful, intelligent flashcards”

If you use emacs org-mode, check out org-drill link . It uses the same algorithms, allows you to tweak everything, and you can keep your flashcards alongside the relevant notes/code/files/issues/ToDos/bookmarks/contacts/feeds/calendar-entry/emails/thesis/spreadsheets/journals/diagrams/pictures/evernotes/etc. link to this video , rather than in a totally separate program.
View on Hacker News

taericon Hacker News • Mentions

On “Patoline: A modern digital typesetting system”

…e upper bounds on editors and such is really that much higher. For inspiration, consider 1. Same for typesetting. Take a look at most of the papers at 2 and tell me whether translating these to new languages is really necessary. (Though, note that I am not arguing against this. Just asking how we plan to know that "things are better now.") 1 link to this video 2 link…
View on Hacker News

fensterbretton Hacker News • Mentions

On “Designing a Personal Knowledgebase”

The workflow proposed in the first link looks quite interesting: Emacs + org-mode + python in reproducible research; SciPy 2013 link to this video And here are some more reviews, opinions, advices and experiences I just stumbled upon: link link
View on Hacker News

daharton Hacker News • Mentions

On “The Many Uses of Org Mode”

I like to use org mode for cleaning up my hard drives... a little script to pipe du into something that sums the files in a directory and adds asterisks, and viola, you have a UI for finding all the large and delete-able files & directories. I haven't used org mode for all the other cool things it can do, but I remember watching a great video about someone who uses it a lot like a Jupyter notebook. link to this video
View on Hacker News