Unless the Singularity has occurred and no one told me, everyone has emotions. I prefer, however, to discuss facts backed up by primary sources, such as the commit stats linked earlier or mailing lists or IRC logs. Do you have an opinion about the situation based on those?
Of course. I'm not making some pie-in-the-sky assertion that arguments should be devoid of emotion, just that I had noticed what appeared enough emotion to overrule civil discourse in the past.
That said, after re-reading your original comment, I don't see much to disagree on (except for one thing, which I'll cover later), and it doesn't seem to exhibit many of the problems I was alluding to, so you have my apologies. It was inappropriate of me to bring that up in this context.
What I should have commented on was that Parrot was all but dead long before MoarVM put the last nail in the coffin. I doubt the announcement did much beyond cement what most developers already knew - Parot was not well suited for it's original purpose, whether you view that as a VM for P6 or a general purpose VM, specifically because for too long it had tried to follow both paths to the detriment of all.
My impression of Parrot, as an outsider who consumed quite a bit of the public information available about parrot development, ended up being that Parrot was too experimental of a VM, and never moved beyond the stage of being a fun sandbox for developers to come experiment in. When P6 was still coalescing around how to implement what was in the spec (and revising the spec in light of that), this didn't matter much. As P6 increasingly moved towards an actual implementation, Parrot's misalignment with what P6 needed along with Parrot's deprecation policy caused many problems between the projects.
I think at one point, prior to MoarVM being announced, there was still a chance Parrot could have been "saved". But by saved, I mean continue on as a sandbox for developers to experiment in, possibly without ever culminating in a VM that was useful as primary target for a language. If enough developers would have been willing to continue with this truth exposed (and I'm not sure enough would), then if Parrot and made the choice to continue as a "general purpose VM" instead of primarily a P6 target, it may have survived as a sandbox.
I'm interested in your opinion on this interpretation - not because I think any of it is news to you, I'm sure it's not - but the opposite, in fact. This view is in no small part informed by your own writings on the subject over the years.
P.S. I was completely serious before when I said I missed your more frequent posting on modernperlbooks.com. I hope that no matter what the future holds for you, that it also includes you writing about technical issues in blog or book form (and enjoying doing so!). I know you've helped solidify my thinking on a great many things over the years, to my great benefit.
I think at one point, prior to MoarVM being announced, there was still a chance Parrot could have been "saved".
The deprecation policy was a point of contention, sure. It was poorly implemented in practice--but it was as much to help Rakudo as it was to reduce the amount of churn in Parrot. Rakudo probably would have had less trouble with it if Rakudo had less code which poked into the internals of Parrot. That's half the fault of Parrot, which had poor design of certain pieces, as well as Rakudo, which rejected a lot of offers to pull some or most of that code into Parrot. (That's a theme.)
I personally spent most of my time on Parrot doing things to improve it for Rakudo--fixing a lot of unpleasant bugs, both in Parrot and Rakudo, for example as well as improving performance. I and other Parrot developers asked multiple times for a list of issues Rakudo wanted Parrot to address. Occasionally we'd get a list and work on them--but I also volunteered to focus on specific improvements for Rakudo and was told "No" or "Not yet" multiple times. You can see that too, if you look in the IRC logs--and not just for things I wanted to do. Andrew Whitworth, in particular, spent months or years on tenterhooks, volunteering to implement the object system called
"sixmodel" in Parrot, but he was always told that it wasn't fully designed yet or it wasn't mature enough. Eventually he left the project too.
On one side, I took a lot of criticism from Rakudo developers for not pushing Parrot harder to be what they wanted and on the other side, I took a lot of criticism from Rakudo developers for trying to change Parrot to be what they wanted. In the face of that seeming contradiction, it seemed obvious to me that they'd already decided to write and maintain a new backend VM from scratch.
My priority--getting a usable P6 release out and stable for general purposes--was clearly incompatible with that decision. Rakudo Star was supposed to come out as a "useful and usable" distribution for early adopters over four years ago, and to my knowledge, it's still only nominally useful and usable for very few purposes.
That is one of my most substantive criticisms of the whole process. Rakudo and P6 have a long history of claiming to be about eighteen months away from general usability, but the project's history is littered with rewrites, questionable technical decisions, and many, many good people leaving in frustration--sometimes quiet and sometimes not.
I begrudge no one working on it, but I believe it's on a path to ever less relevance without a dramatic change in project management. I'll reluctantly take the blame for mistakes I made as a member of the P6 design team and as a leader in Parrot, but I object to rewriting history to suggest that Parrot was solely at fault for Rakudo's current state.
I understand you were in what seems an untenable position between Rakudo and Parrot. I'm not trying to further the "Parrot caused all Rakudo's problems" argument, just my own which is that Parrot and Rakudo were destined to either split farther or merge at some point. It's unfortunate that Parrot started towards what appeared to be a merge at just about the same time Rakudo decided a deeper split was the only way forward.
Personally, I've always viewed MoarVM as a spiritual successor to Parrot. Parrot and Rakudo begat NQP, and NQP has survived parrot. It's a shame more parrot devs didn't jump to MoarVM, but they of course had their own motivations for working on Parrot in the first place, which may not align well with being "the Perl 6 VM" (among any number of other reasons), which MoarVM exemplifies more than Parrot had in many years.
It's a shame more parrot devs didn't jump to MoarVM
Why would they? Rakudo's position was "Parrot is fundamentally broken and Rakudo is actively moving away from it." You can see how Parrot's development all but stopped within a few weeks of that discussion from the Ohloh link I posted earlier. More than that, Moar's development began in secret around that time, so how would Parrot developers know that they should have been working on that instead?
(Rakudo developers often claim that Parrot had no singular focus on Rakudo, but that graph argues differently to me. Then again, Rakudo developers also claim that they decided against using Parrot because Parrot development ceased, but that's an obvious post hoc ergo propter hoc argument to anyone who looks at the chronology.)
Personally, I've always viewed MoarVM as a spiritual successor to Parrot.
I looked at the code briefly after its announcement. It seems to repeat most of the long-standing architectural flaws of Parrot and it ignored much of the design work that we had been doing to make a Parrot which could compete favorably with fast VMs such as LuaJIT or v8. (My impression then was that, if Rakudo hadn't chased away Parrot developers and Parrot were free to ignore pesky things like backwards compatibility, deprecation, and users who wanted it to continue to work, Moar looked a lot like Parrot would have after the same time period. It was pretty disappointing.)
> Rakudo developers also claim that they decided against using Parrot because Parrot development ceased, but that's an obvious post hoc ergo propter hoc argument to anyone who looks at the chronology.
Maybe look at the chronology again?
> {MoarVM} seems to repeat most of the long-standing architectural flaws of Parrot
Such as?
> (Moar looked a lot like Parrot would have after the same time period. It was pretty disappointing.)
As you now hopefully realize, the time period was 14 months, not 29.
You mean the first commit committed to a repository which survived long enough to be made public, eventually. Even so, if you look at the timestamps on those first commits, you realize that either the design of the VM leaped from someone's head fully-formed like a virtual Athena, or (as was widely known at the time) that someone had been designing and playing with ideas for much longer.
Maybe look at the chronology again?
The one where Rakudo developers told Parrot hackers not to implement sixmodel to replace Parrot's default object system (the one which one of those Rakudo developers had, in fact, actually implemented) during the period where it was obvious that those Rakudo developers were in fact designing their own VM?
The one where Rakudo developers told Parrot hackers not to implement sixmodel to replace Parrot's default object system (the one which one of those Rakudo developers had, in fact, actually implemented) during the period where it was obvious that those Rakudo developers were in fact designing their own VM?
Hmm, I read that as Rakudo devs trying to keep Parrot devs from starting a project that they may not want to continue in the future (given your statement it was obvious they were looking into developing their own VM). If the decision has been made, trying to reduce fallout seems wise to me, given also that they weren't public on the new direction, which I don't know the details of why.
I'm not sure that changes the equation from where I'm standing. When you want to keep someone from wasting time but can't/won't explain why, using "hold off"until you can finally explain seem the logical choice since "don't do that" gives away too much.
Again, I'm not addressing the given that information needs to be kept hidden, I don't know the reasoning for that. It just seems odd to me that this gets singled out as a negative when I see it as trying to make the best of a bad situation.
I'm only following up on this because it seems an apt example of the emotionally driven arguments I was referencing before. I see at least several possibilities. 1) There is information I'm don't know or am not inferring correctly, 2) we have some mismatch in values that is causing us to interpret the same situation differently, 3) your emotional context causes you to interpret the situation differently (non-rationally), or finally, I have to admit it's possible 4) my emotional context is causing me to interpret the situation non-rationally.
The outside appearance to me, as stated previously, is #3. That of course in no way makes it the most likely answer. (Sorry if you find this boring, I find it interesting as it mixes two interests of mine).
I'll try to be as clear as possible, at the risk of sounding like a humorless pendant.
When Rakudo announced it wanted to rewrite NQP to run on multiple backends, the stated justification for not relying on Parrot in the long term was twofold. First, because Parrot developers didn't treat Rakudo as the most important hosted language. Second, because Parrot didn't provide the features Rakudo wanted--in particular, its object model was unusable by Rakudo.
Those reasons are tied together; the second was used as proof of the former. You can also throw in the deprecation policy as a supporting reason, but that makes things more complicated because Rakudo developers wanted it in place for some things ("you can't change things in Parrot and break our code") and wanted it gone for other things ("why can't you just fix this thing?").
I have a problem with both reasons #1 and #2, because I have multiple examples of Parrot developers offering to do make changes to help Rakudo. In my case, I was told not to do them. In the case of sixmodel, the stated impetus behind #2, Andrew and others (who had been accused of not wanting to help Rakudo) were continually volunteering to write the very code that Rakudo developers said they wanted and were continually told "No, not yet." (See Raiph's links, for a few of many examples. See the #parrot and #parrotsketch logs for many more.)
Meanwhile, Rakudo developers were continually complaining how Parrot developers were not interested in helping Rakudo and how Parrot was technically a bad fit, citing sixmodel as an example.
I interpret that response as something other than good faith. I believe that's why Parrot developers left; there's no point to sticking around in that situation.
Thanks, that does explain a lot of your reasoning more clearly. I'm not sure I can change my assessment of the situation though, as the relative timings of these events, which I think matter greatly, aren't known to me, and the Rakudo interpretation of these circumstances isn't known to me either. I believe the Rakudo response can be explained by their belief that the special relationship between projects was unsalvageable, and the was forward was no longer in Parrot. In that case I try to fall back on my default, which is to assume people will act in their best interests while also trying to minimize damage and discomfort to others unless given cause.
Whether the actions of the Rakudo developers indeed did cause a major Parrot exodus is something I'll easily cede at this point. Whether that was intended, unintended or the opposite of what was intended seems to be what we are really talking about here (because I believe intention can matter, not in assignment of blame, but as possible mitigation of scale).
The irony of this is that Parrot seems to now well and truly be a P6 focused VM, as the only work done on it (besides rurban and util) is by Rakudo devs to fix bugs or make small changes to new Rakudo stuff works. Why rurban still bothers is beyond me, AIUI he's still got p2 to work on.
In any case, you've been more than a good sport in humoring me in all this; you put up with more from me than I would have imagined. Thanks!
Whether that was intended, unintended or the opposite of what was intended seems to be what we are really talking about here.
Let's assume the Rakudo developers acted in good faith per your reasonable default. The effect is still that Parrot is all but abandoned, multiple productive developers with decades of practical experience attempting to implement P6 no longer contribute to either Parrot or Rakudo, and Rakudo isn't obviously more usable than it was three years ago. The effective delivery date of P6 is still "some time in the future". The fourth anniversary of Rakudo Star is approaching and it hasn't met its goals.
Even if Rakudo were released in a stable, 6.0 form today, I still wouldn't use it for anything I care about because I don't trust its developers to deliver usable software reliably.
Rakudo isn't obviously more usable than it was three years ago
I don't believe that to be true. Even if by usable you mean primarily can be used in production, I would still argue it's more usable that it was, even if I wouldn't label it strictly "usable." I wouldn't shell out to bc for math ops, for any number of reasons (including it being slower for my purposes), but I wouldn't call the bc utility unusable.
I don't trust its developers to deliver usable software reliably
I can't fault you for that conclusion, given your experiences. That said, developers involved in a project change over time, as does the influence of existing developers (and maybe you believe people can change, too). That conclusion may become more or less true over time, and indeed may have already changed. Perhaps some day you'll find a reason to be interested again, and find the experience more enjoyable. We can only hope. :)
TL;DR Imo specific evidence directly contradicts chromatic's assertions about the intentions, plans and actions of the Rakudo dev leads (Patrick and jnthn). My assessment, and chromatic's too I think, is there was a growing breakdown in trust that blew up in 2011 with serious consequences for both Rakudo and Parrot.
> When Rakudo announced it wanted to rewrite NQP to run on multiple backends
What announcement are you speaking of? In the past you seem to have suggested they announced this in late 2010 or early 2011. Available public evidence, including a github repo, makes it clear that one of the primary objectives of the 2009 NQP rewrite was to transition to a multiple backend architecture, and there was no attempt to hide this.[1]
There were discussions in early 2011 about the next stage in the multi backend evolution, and about elements of Rakudo's latest NQP becoming a central part of Parrot, and this did seem to get derailed by misunderstandings about NQP's support for multiple backends. Maybe that's what you're talking about?
> First, because Parrot developers didn't treat Rakudo as the most important hosted language.
Patrick's stated justification for a multiple backend Rakudo is explicitly grounded in Larry's 2001 specification that Perl 6 must run on existing major VMs[2] and Patrick's decision when he became the Perl 6 project manager in June 2009 to prepare Rakudo to run on JVM and CLR.[1]
That said, by the time Rakudo moved out of the Parrot repo in early 2009 there were definitely major problems with the way Parrot and Rakudo dev meshed (or not)[3] so it might have been good if Parrot had treated Rakudo as an important hosted language, or at least had better responded to what Rakudo actually asked for, according to the mailing lists and IRC logs from 2009 thru 2011, namely for there to be better management of breakage.[4]
> Second, because Parrot didn't provide the features Rakudo wanted
Again, aiui, this wasn't a stated justification for Rakudo considering multiple backends before 2009 nor was it when Patrick rewrote NQP with support for multiple backends in 2009.
But, yes, the growing gap between what Rakudo was saying it needed from Parrot and what Parrot actually provided must surely have led Patrick and jnthn to ponder a plan B, and spending a few years writing another VM would have been a natural thing to consider in the midst of the deteriorating Parrot situation in 2011 despite the very scary prospect of yet another huge delay in getting P6 to 6.0.
> I interpret {Rakudo's} response as something other than good faith. I believe that's why Parrot developers left; there's no point to sticking around in that situation.
There were indeed clear signs of a severe breakdown in trust littered throughout 2011.
I think jnthn and especially Patrick really tried to solve this problem but were fighting a losing battle. I acknowledge my bias (I like Patrick and jnthn) but there are tons of examples such as Patrick drafting a formal Parrot/Rakudo relationship policy in mid 2011:
I agree there would be no point in folk continuing to cooperate if there's an assumption of bad faith. Typically if person/group A interprets person/group B's actions as being done in bad faith, person/group A themselves starts acting in bad faith, or at least in a non-cooperative manner. If this attitude spreads throughout a group then all is indeed lost.
----
[1] The two Rakudo devs who matter in this context are Patrick and jnthn.
Patrick Michaud's 2013 "Perl on JVM" talk made it clear that one of the goals of his 2009 NQP rewrite was to restructure it to support multiple backends corresponding to existing major VMs (with JVM and CLR specifically in mind).
Googling for rakudo multiple backends with a date filter and then doing a quick archive.org dig shows that sometime in 2009 jnthn changed the "What do I do in Perl 6?" section of his about page to include "Transforming Rakudo from a single backend compiler to one capable of targetting multiple backends".
[2] "Perl 6 must not be limited to running only on platforms that can be programmed in C. It must be able to run in other kinds of virtual machines, such as those supported by Java and C#." from http://www.perl.com/pub/2001/04/02/wall.html
Again, please leave me out of your uninformed and biased speculations. You weren't there. You weren't involved. I was--not only as a developer on both projects but as the P6 project secretary. In that capacity, I took and published hundreds of pages of notes, and I'm more than capable of speaking for myself.
I noted what I thought was a parallel between my assessment ("breakdown of trust") and yours ("bad faith"). You may not see the connection and I may be wrong but my comment was for all readers, not just you, and was qualified with "I think".
My assessment was not uninformed. As you've said yourself, it's all there online and I've thoroughly researched the relevant period.
The key exception is that I have been unable to find your published P6 call summary notes. I would appreciate a link to them.
> if you look at the timestamps on those first commits
jnthn churns out great design off the cuff and generates rapid sequences of brilliant and clear commits. I've been following #perl6 for a couple years so I've gotten used to it. (I wasn't surprised when I recently found out he has a first class honors degree in cs from cambridge.)
That said, confirmation bias is an ever present danger so I still investigated a lot more than just the timestamps on the first commits before I posted my prior comment.
Imo the pacing and content of the repo's early commits (all jnthn) are typical for jnthn.
> Rakudo developers told Parrot hackers not to implement sixmodel
The period you're talking about is early 2011 thru early 2012.
The only two Rakudo devs that are relevant in this context are jnthn and Patrick.
If jnthn was telling Parrot devs not to implement 6model throughout this period, why, on May 31st 2011, did he write several 6model docs in response to a request by lucian? Why didn't he just tell lucian not to implement 6model?
If Patrick was telling Parrot devs not to implement 6model, why, on July 6th, 2011, did he suggest a two month delay of doing so with whiteknight? Why didn't he just tell whiteknight not to implement 6model?
On february 2012, two months before jnthn started the extant MoarVM repo, not_gerd wanted to know some stuff so he could implement 6model. Again, why didn't jnthn just tell him not to implement 6model?
I wasn't trying to imply any sort of wrong decision on the part of Parrot developers for not working on MoarVM. I tried to make sure that was obvious. Perhaps I failed.
More than that, Moar's development began in secret around that time
I'm aware, which is one of the things I was alluding to when I mentioned "among other reasons" for not working on MoarVM. Beyond not knowing it existed, when they did there was probably a feeling of betrayal. I'm not trying to ignore that. The shame is that some of these people would most likely have enjoyed working on MoarVM if things had played out differently.
My impression then was that, if Rakudo hadn't chased away Parrot developers and Parrot were free to ignore pesky things like backwards compatibility, deprecation, and users who wanted it to continue to work, Moar looked a lot like Parrot would have after the same time period. It was pretty disappointing.
That actually sounds like praise to me (barring some of the architectural missteps you allude to earlier in your statement). That MoarVM was able to mostly get to where you think Parrot could have been if it had been able to ignore a lot of extremely hoary and cumbersome problems seems like a good thing. Did you find it disappointing because of MoarVM specifically, or because of the realized loss of potential from Parrot?
In the end, if the Parrot split solved some P6 roadblocks, and spurred more rapid P6 development, which I believe it did, I have to count the action as a wide choice. I found Parrot interesting in it's goals, but it's target of being a VM for scripting languages was starting to get competition from the JVM, which I'm not sure it could compete with. P6, on the other hand, still hasn't seem a competitor feature-wise IMHO to make me think it's future is in doubt from an outside source (whether it will ever be successful, however you define it, is a different question). Of course your level of investment and interest in each of the two projects may cause you to weigh the situation entirely differently.
Did you find it disappointing because of MoarVM specifically, or because of the realized loss of potential from Parrot?
Several reasons, in no particular order.
One, because of the deliberate driving off of Parrot developers and their knowledge about what works and what doesn't work for building a VM for a Perl. (Sure, you can believe that I feel a little personal betrayal there, but there's also a concern that the Rakudo developers were taking on yet another project for which the bus number is, as usual, abysmally low.)
Two, because the resulting design (when I looked at it) was adequate, at best. I believe it won't compete with fast VMs without a serious change in its architecture, and no amount of magical thinking about how a GSoC student will build a JIT in 10 weeks will fix that.
Three, because it represents yet another example of NIH thinking and throwing out working code in favor of spending even more time building something new from scratch.
Four, because it set back the P6 release date by at least 18 months, if not years.
Five, because all it represented when I looked at it was the kind of code shuffling that Parrot could easily have done in the past three years, without doing anything much else (such as make the architecture improvements we'd begun to design and prototype).
One fact often forgotten in all of the nonsense about how "Parrot lost its focus and thus its purpose" is that Parrot was, from the start, intended to run at least both Perl and P6 simultaneously in the same process without embedding libperl.so.
