Skip to content
Hallway
0:00 / 0:00

#haskell#garbage-collection#parallelism

ZuriHac2016: Parallelizing and Distributing Scientific Software in Haskell

Google TechTalks

349K subscribers • 62 videos on Hallway

YouTube
  • Recommended by 2 people: 2 in engineering communities.

Advanced. Deep dive for people who know the field.

2.6K views10 years ago2 recommendations

Programming languagesSystems programmingHaskellConcurrency and parallelismPerformance and optimization

Francesco Mazzoli describes the challenges and workarounds of parallelizing and distributing a CPU-intensive Haskell application across many cores.

A Google TechTalk, July 22, 2016, presented by Francesco Mazzoli ABSTRACT: Suppose you have a pure, CPU intensive function that you need to run on 10000 different inputs. This seems the dream task to parallelize in Haskell, but as it turns out it is not as easy as one might think. In this talk I'll describe what we needed to do to parallelize such an application on machines with as many as 18 cores, and further what we needed to do to distributed the same application across separate machines, on hundreds of cores. wiki.haskell.org/ZuriHac2016

Why it's here

2 people recommend this

What people say. Highly recommended for understanding Haskell's multicore scaling limitations and stop-the-world garbage collection bottlenecks.

Moments people linked to

u/metafunctorin r/haskell • Recommends

On “Techempower Benchmarks #14 Released with Servant, Yesod, Spock and Snap”

Without being an expert in libraries for JSON serialization or database access, I think that these libraries have very poor performance and must be improved. Specially database access, that comes from old libraries. Web servers have been frequently benchmarked and optimized, so they are not the problem. Another big problem is latency and the huge amount of garbage collection that the groped stop-the-world GC engine has to perform. That is aggravated when the number of cores increase. See this. A pure language like Haskell generates a huge amount of garbage. Managing this with a stop-the-world collector seems to be a particularly bad idea for performance in these kinds of server applic…
7View on Reddit

u/agocoronain r/haskell • Recommends

On “Synchronous time bound GC”

Does it solve this problem? link to this video Then it is badly, badly needed!
5View on Reddit

u/metafunctorin r/haskell • Recommends

On “Disadvantages of purely functional programming”

The original post is old but he is mentioning good points and they are still real concerns, specially the lack of scalability when the number of cores increase. This is due to the stop-the-world gargage collection. This people not suspicious of being trolls (FPcomplete) depict the problem very very clearly so anyone can understand: link to this video In order to scale better, they have to resort to create many processes (a process per core) in the same machine and managing it like al distributed system. That means that the problem still exist
5View on Reddit

u/agocoronain r/haskell • Recommends

On “State of Machine Learning, NLP, etc in Haskell currently?”

…reet, your stops on average are a fourth smaller. The gain/loss is more acute when the number of cores increase. The problem can be reducing using pre-allocated mutable data structures, but it is not ever possible to make the allocation minimal. This presentation explain the problem with multicores and stop-the-world garbage collection of Haskell link to this video…
2View on Reddit

u/jmhimarain r/haskell • Mentions

On “Monthly Hask Anything (August 2021)”

In this link to this video from 2016 the speaker talks about the difficulty of writing large parallel (mostly scientific) software in Haskell because the garbage collection becomes a significant bottleneck when running multiple cores. He also talks about how they had to come up with their own workaround, which worked fine but was kind of a pain to implement. Has this been addressed in more recent implementations of Haskell, or is it still an issue?
2View on Reddit