My latest attempt at boiling it down is to say that Darcs performance is about three things: (i) theory, (ii) engineering and (iii) implementation.

Theory means the core theory of patches which makes Darcs powerful and easy to use… until you hit a conflict. Simple conflicts are fine. When conflicts nest, however, you do get a combinatorial explosion of nested “merger” or “conflictor” patches. The darcs-2 version of patch theory obviates a lot of this nesting, but we still have a lot of work to do on the theory end. Haskell is irrelevant here.

Engineering refers to high-level choices about how we store information and ship it around. Darcs performance has been improving a lot in the last two years because we have been emphasising the engineering aspect of things. For example, we use timestamps to determine if we need to check the committed and working copy of a file for differences; however, because we also use hard-links to save time/space, the timestamps from the filesystem can mislead Darcs into checking for way more differences than it needs to. Recent versions of Darcs use an index file to store our own timestamps. Another example of work we are doing in this line is to “pack” our many small files into fewer larger files that we can transfer over HTTP a lot more quickly.

Implementation can include choices like Haskell as a programming language. The last time I tried to clarify this, I said that Haskell doesn’t make a difference here. Thinking more about it, I realise I was likely wrong. Sorry! One place where Haskell *could* make a difference is that we have a number of performance bugs we have open where Darcs seems to be using an unreasonable amount of memory for the task at hand. It’s not yet clear to me if these are actually engineering bugs in disguise, or if some of them are not merely implementation details. The Darcs team is quite small; we need all the help we can get. Once we’ve reached a couple of engineering milestones we have in mind, it’d be really great if the wider Haskell community could give us a hand on understanding these bugs.

Meanwhile, I still have the conviction that solving a lot of the engineering issues (Darcs 2.5 and 2.6 look very promising) will make Darcs a lot faster for most people, most of the time. We still need to work on theory and implementation over time, but I hope you’ll like what we’ve accomplished for the short term.

“I disagree with point 8. Performance in Haskell is no less predictable than
in any other enterprise class language when in the hands of a programmer
who is really thinking in Haskell. The real issue is 3 again – it’s different.”

How well is Darcs doing on containing sudden, unexpected, time- and space-consumption combinatorical explosions again? Oh, right. The Darcs people are just not thinking in Haskell….

If you look at those more closely, they often follow a pattern.
These posts tend to be from experienced and successful
imperative programmers with a fair amount of hubris, sometimes
well-deserved and sometimes not. They expect to conquer the
entire Haskell community within days, as they have done in other
languages, so they jump right in to some very ambitious project.

When they soon find themselves in much deeper waters than
they bargained for, the reactions vary. The easiest reaction,
especially for someone with hubris, is to blame the language.
A few were so psychologically scarred that they became
pathological anti-Haskell trolls to this day. Others are capable
of realizing that Haskell is just different – they go on to be
enriched by the experience, whether they continue with Haskell
or not.

Aside from the above scenario, the next most common kind
of post about performance issues is in cases where the
underlying problem is difficult. These issues will of course be
more or less the same in any language.

Finally, you are left with actual language design issues relating
to performance. Decades ago, when functional programming was
new, there were common cases that were difficult performance
problems in functional languages while trivial in imperative languages.
Those issues have long been settled. Now the constant process of
gradually increasing performance looks about the same in Haskell
as in any other mature language – though a pure functional language
does seem to have a decided advantage for emerging platforms.

Then there is the theoretical side. Again, decades ago it was
sometimes difficult to state and prove results relating to
performance in pure functional languages, because much of the
existing literature was formulated in strongly imperative
language. Now much progress has been made, but there are
still open research questions. I’ll leave the literature review to
the academics among us.

In general, whenever the trolls start piling up “proofs” that
performance is an inherently difficult problem in Haskell, trace
back the actual sources. You’ll find that in each case, either
the person complaining of performance problems is
an inexperienced Haskell programmer, or the source is at
least a decade old.

Ultimately, the myth that there are inherent performance problems
with pure functional languages stems from the fact that they are
indeed quite different from imperative languages. I think it is
important to debunk that myth.

@dons I can’t really speak on behalf of the original author, but the RPC example aside, I thought his point was essentially that we have Quantity (i.e. breadth), but not the Quality? I’m not sure how fair the “Outside World” point is – I have had problems along this line but mainly due to the Windows issue.

@Yitz Fair point. I’m not really sure it’s cut and dry either way. I don’t think the situation is the same as imperative languages such as C/C#/etc though:

On the one hand Haskell does far more aggressive program transformations than, say, a C++ compiler will do. This is a bit of double edge sword – I want this power as it’s one of the major benefits of a pure functional language, but it does increase the amount I need to learn. Haskell in GHC is even more unique in that we can specify our own code rewrite rules. This is very cool, and although it’s not as if I can ignore the implications of this, it’s still a major plus in my book. I guess it’s another way of saying powerful tools are powerful .

On the other hand, when we get it wrong and introduce, say, a space leak, the difference in performance you can get can be pretty staggering. Space leaks in particular can be really quite a pain in the ass. There’s a fairly regular stream of posts on the cafe from people asking for help figuring out what’s up with their code.

When the performance needs are moderate, a native Haskell speaker
will instinctively avoid the kinds of constructs which tend to cause space
leaks and performance sinks. If such problems do occur, it won’t be
harder to recognize where they crept in than in any other language.

When performance needs are heavy, there is a well-stocked optimization
toolbox. Using it may require some trial and error like in any language,
but no unusually wild guesswork is needed.

But a Java or C++ programmer who is valiantly trying to write
idiomatic-sounding Haskell may indeed find that certain small changes
in the code cause what seem to be magical effects, for better or
for worse, in performance. Just as a hypothetical Haskell programmer
who has never heard of imperative languages before would make the
same observation about Java and C++.

And that, of course, leads back to 6, and the false perception of 4.

> If you want to connect to an RPC server and fire off a few procedure calls… good luck with that. Same deal for talking to databases, accessing OS resources, manipulating binary file formats… You’ll basically have to write everything yourself from scratch. This is a big deal for commercial work.

We actually have more than 2000 libraries on Hackage,

http://hackage.haskell.org/packages/archive/pkg-list.html

Including half a dozen RPC libraries, more than 50 database components, including all major server and cloud databases, and hundreds of “system” libraries.

I’m really struggling to see library breadth as a problem.