Jan Vitek - Getting everything wrong without doing anything right!

And the lesson is that data must be cleaned. It's highly unlikely that the average C ~~file~~ repository has 15610 ~~lines~~ files. And you can clearly see that most outliers are autogenerated or compiler test cases. [*Getting everything wrong without doing anything right -- on the perils of large-scale analysis of GitHub data*] is a great talk about the pitfalls of automatic code repository analysis.

…with adaptive technologies when they actually come. Put simply, saying you ain't blub ain't enough to make it so. > Though I suppose we do know from that paper that with 99% certainly that highly-starred open source Clojure repos on Github have at least 20% fewer bug-fixing commits than highly-starred open source Java repos on Github We don't. (link to this video)…

On the perils of large-scale analysis of Github data

It sounds like Philip Wadler is in the audience and gets into an argument with someone else in the audience who is mostly inaudible. Can anyone make out what the inaudible guy is saying? Where Wadler asks his question. The response shows up soon after.

> Until you show me data to the contrary, the only rational way to choose is to assume that there's a difference, like most things in life. That belief doesn't help you choose. If there must be a difference, then why believe that your favorite Foo is best? Maybe it's the worst and so you must *not* choose it? You're free to believe what you like, and ultimately, like me, you'll choose what feels best, but your belief that something has a real universal effect without substantiation is what science calls delusion. In science, you must accept the null hypothesis when there's lack of data (and no model). Otherwise, you cannot know which of the thousand conflicting and equally unsubstantiated…

It's not *the* journals, as some rejected the papers, but *some* journals (i.e. *some* reviewers). And I would find it very surprising if researchers in *any* discipline were not biased as a result of studying one side of the story and their conclusions skewed. You see it in biology, paleontology, physics, and also in computer science (e.g. I see a huge bias among people who research software correctness depending on whether they come at it from programming language theory or formal methods theory; I am frankly amazed at the bias I see, and that's just the one discipline I am relatively closely familiar with). Just yesterday I posted to this sub this video about a very good reproduction st…