u/agumonkeyin r/compsci • Recommends
The tension between convenience and performance in automatic differentiation – Jeffrey M. Siskind
0:00 / 0:00
#automaticdifferentiation#autodiff#compilers
Jeffrey M. Siskind – The tension between convenience and performance in automatic differentiation
MILA: Montreal Institute for Learning Algorithms
203 subscribers • 1 video on Hallway
- Recommended by 2 people: 2 in engineering communities.
- Presented by Jeffrey M. Siskind, Purdue University
- Linked in the descriptionSlides on autodiff-workshop.github.io
- May be out of date. Uploaded 9 years ago, in an area where tools change fast.
Advanced. Deep dive for people who know the field.
714 views9 years ago2 recommendations
Systems programmingAI and machine learningCompilers and interpretersNeural networks and deep learning
Jeffrey M. Siskind discusses the trade-offs between convenience and performance in automatic differentiation systems.
[NIPS 2016] Autodiff Workshop Speaker: Jeffrey M. Siskind, Purdue University Slides: autodiff-workshop.github.io/slides/JeffreyMarkSiskind.pdf Workshop: autodiff-workshop.github.io/2016.htmlWhy it's here
2 people recommend thisMoments people linked to
u/abstractcontrolin r/Compilers • Recommends
Jeffrey M. Siskind – The tension between convenience and performance in automatic differentiation
u/abstractcontrolSpiralin r/ProgrammingLanguages • Mentions
On “On Machine Learning and Programming Languages”
Here is a talk by the author of 3. I've posted it twice already on Reddit recently so I might as well do it again as it covers my two favorite areas: machine learning and partial evaluation. Staging, which is user directed partial evaluation really covers a lot of the points you've outlined above and it is only a matter of time until the ML people wake up to how useful it is.
u/abstractcontrolSpiralin r/ProgrammingLanguages • Mentions
On “On Machine Learning and Programming Languages”
> How would it help with machine learning frameworks? Does it help with symbolic differentiation? Apparently if you use Python's operator overloading, there is no problem with symbolic differentiation. It is used heavily in "real" code.
Towards the end of his talk he gives a few performance benchmarks between his language (Vlad) and the rest. Operator overloading is a common way to implement autodiff in a language as a library, but it is very inefficient. With sufficient PE/staging that problem goes away.
Describing that as the main motivation for having staging though is not entirely satisfactory to me though. It is true, but if it was just that I would not have bothered making my ow…