CAs have a lot of nice properties: translation invariance, locality, arbitrary population sizes, parallel computation, simplicity of implementation, and emergent complexity. Interesting to see where this line of work goes
When I draw an 8 it seems to frequently classify it as a zero. In fact, most of the numbers I draw don't get correctly classified. Just the 0's and 1's are consistent.
I found the same thing. If you draw an 8 with more of an 'x' in the middle, it's fine, but if you draw two o's on top of one another, it can't figure it out. I suspect they need to increase the diversity of their training set.
I really enjoy how quirky Alex, Ettore, et al's experiments are. It isn't going after benchmarks. It has a very "what would happen if we tried this?" feel to it. If I had to bet, I don't think it's super likely to have direct real world applications, but it's immensely fun to read about and expands the space of ML systems I might imagine.
I've also been enjoying the open slack channel associated with this thread. It's pretty low volume, but there's been a number of fun experiments from people who enjoyed the original article on neural cellular automata. I feel like there's often a lot of really cool experiments and conversations that no one outside a given lab sees, because they don't make it to publication, and I like how the channel allows those to be shared with anyone really excited about this line of work. It was also really heartwarming to see people jump in to give feedback on the draft of the most recent article!
Hi there! Like this line of work. As an enthusiast of Alife-ish things, CAs (and NKS-style computational systems generally), and also an author of an ML library, I was looking forward to jumping on the slack and just having a chat with likeminded folk, but I'm having trouble finding how to actually make an account on https://distillpub.slack.com
It's better to pick the most interesting specific article and link to that, instead of posting lists. Otherwise the topic is a bit of a greased pole - there isn't much to grab onto and the discussion ends up being about the lowest common denominator.
Alternatively, if the intended topic is the experimental discussion format, that's fine, but then it should explicitly be about that rather than the content that happens to supply the example.
Thanks for your comment! I'm one of the editors of Distill.
I understand the task to be different from regular pixel segmentation in that it's set up in the cellular automata context and message passing is only allowed between neighbors. I think this is what the authors intend by "each pixel in the digit needs to coordinate locally".
With that said, you bring up a good point that it could be misread. I bet the authors would be happy to update the paper and clarify if you wanted to engage!
The problem is that HN is an internet forum, not scientific discourse. When you vent what looks like garden-variety bile into an internet forum, all we get is a shittier internet forum, not better science.
No doubt a colloquium of colleagues in a scientific department can handle much more blunt and even ferocious feedback than a large broadcasting channel like this place. It is a question of what each medium is good for, what sort of conversation it can sustain. Whether a message has a good signal/noise ratio depends on the medium.
Put another way: you need to know which game you're playing. In some games, knocking an opponent to the ground with maximal force is understood by everyone to be a great move—including by the person you tackled—while in other games, the same action can get you thrown out of the game, or out of the league, or sent to jail, or even killed. It can also destroy the game itself.
In the game here, things like vitriol and snark are associated with low-quality messages, because so many people do it all the time, and eagerly take others doing it as license to do worse. Usually it is quite off topic and devolves into repetitive brawls. In such a context, you can't communicate interesting information that way. The baseline of noise will drown out your signal.
To communicate effectively here, you need to take a different approach: when you know more, share some of what you know so the rest of us can learn. If something isn't good, you needn't pretend that it is, but don't be gratuitously negative. Don't punish or shame anyone for being wrong; instead, provide correct information neutrally. If someone is learning, help them learn more. These are strategies that work for high-signal communication here, and they have the beneficial side effect of building up the context for successful future conversation, rather than breaking it down.
The cost of doing this is a certain blandness in communication which is regrettable, but I don't know how we can avoid that.
It's clear from your now-deleted comment upthread that you have real knowledge in this area and I hope you will continue to contribute it. Please feel welcome to post another comment if you still want to, and no need to feel bad. We're all trying to figure out together how to have an internet forum, in the long run, that doesn't suck.
Hello! I'm one of the editors of Distill. I'm certainly open to the possibility that we could have helped the authors differentiate their contribution more.
It seems like it's important to you to call Distill out on what you see as an institutional failure. I think you could actually do that more effectively and credibly if you wrote your comment differently.
Putting myself in the author's shoes, it feels pretty hard to engage with your comment. When you swear, it gives the sense you don't wish for people to engage with you. When you say "who the fuck reviewed this paper, a 25 year old fresh professor", it feels like you're implying that people who are young are intrinsically not knowledgable. When you don't leave open space for more charitable interpretations, it makes productive conversation a lot harder.
I appreciate your dedication to improving scientific discourse. I'd love to see if we can turn this into a productive conversation.
I took my other comments down, because I realized I was being ignorant to the real people who wrote the paper/operate the journal, and I apologize for that. I think other's have criticized Distill much more aptly then I ever could, so I don't think I have much to contribute to the conversation
Thanks for the response! Your apology makes me feel more positive about conversations on the internet. :)
I genuinely appreciate you giving feedback. Running Distill is hard, and there's lots of places we make mistakes or could do things better.
From my perspective, Distill has to balance a lot of equities. I see one of our major roles as trying to explore new visions of what an academic journal can be, so I put a lot of weight towards us doing weird and unusual things, like these threads. But we also need to take care around process and quality. And sometimes we screw up.
Distill is also entirely volunteer run. Everyone involved has full time jobs, and runs Distill on top of that. And honestly, we're often quite exhausted. So sometimes we make mistakes that a better version of ourselves with more time for Distill would not.
I really appreciate the idea of a place where code and paper can be intertwined, it's a cool idea for a journal! I also really enjoy journals doing weird unusually things! I just have a penchant for people defining novel terms and not fully exploring existing literature in a field. In my opinion, it creates a very disorienting effect where those who have useful tools to contribute (typically neural network researchers) immediately start on the worst footing.
Everyone is constantly playing catch-up in understanding and can't follow the evolution of an idea/methodology over time.
I was under the impression that Distill was funded by Google, is this not the case?
Nope. There's money committed if we ever run the Distill Prize ($10K prizes recognizing outstanding scientific communication), but that's tied up in legal issues.
For regular Distill, all the costs (several thousand dollars a year) come out of volunteer pockets, and all the time is volunteered.
Edit: There was a period of 6 months where Firebase (which is connected to Google?) gave us free hosting, reducing our expenses.
That's impressive, and also re-contextualizes a lot of my feelings towards the journal. Have there been any plans to go more mainline in funding efforts. I know all open access Nature journals are "pay to publish". It's a slippery slope to being predatory, but without university's paying for subscriptions, the funding for a journal must come from somewhere and Nature seems to maintain quality in spite of their model (in this case donations).
We've sometimes discussed it in the context of whether we can "scale up" Distill. But I think at this point I expect us to just stay niche and volunteer focused.
Based on one of your twitter the calculated IF of Distill is 23. So if you're journal is reaching so many people, wouldn't moving towards more robust and professional editorial staff be beneficial for the large audience? Or do you think there's a benefit too having volunteers?
> if you're journal is reaching so many people ...
Just to clarify, impact factor measures the average impact of a paper, not the total impact of the journal. In fact, it divides by the number of papers! So it's much easier to have a high impact factor if you're very small, and it's probably not quite the right way to think of whether society should invest resources in something.
(In fact, we technically don't have an IF, just the value that would be an IF if we were bigger. IF isn't officially defined until you reach a certain number of publications a year.)
> wouldn't moving towards more robust and professional editorial staff be beneficial for the large audience? Or do you think there's a benefit too having volunteers?
I think there's two main things.
Firstly, it's just a very difficult transition to make. If we wanted to hire staff, we'd need to find people who were aligned with Distill's vision and wanted to dedicate their carer to it. Hiring is hard! We'd also need to find a reliable funding sources (probably at least 200K/year) that was reliable (we don't want Distill to suddenly disappear!) and dosn't impose negative externalities (like being closed).
Secondly, it's a lot harder to experiment when you're big and have commitments to more people. It's already quite hard for Distill to do weird things now -- a lot of people get annoyed to us, and I feel a visceral pressure to be conservative.
Also, a lot of our ability to do weird things and have beautiful articles comes from expert attention to individual papers. A lot of early Distill papers involved Shan Carter or I spending 50-100 hours mentoring authors and giving them design and writing feedback. There's no model where that make sense at scale.
>Secondly, it's a lot harder to experiment when you're big and have commitments to more people. It's already quite hard for Distill to do weird things now -- a lot of people get annoyed to us, and I feel a visceral pressure to be conservative.
I wonder who those people could be, lol.
It's interesting to focus on a work that doesn't function at scale, it seems antithetical to a lot of common knowledge that I often see about the state of the world. I guess this sort of cuts at the same debate which happens at a lot of universities, which is how many students to allow to take a class. I am a very firm believer in having hundreds of niche opportunities to engage instead of one monolithic method of teaching/engagement (and I provide undergrads at my uni the opportunity to work with me and my group for similar reasons).
But it's difficult when the benefits of something are so strong and can lead to direct financial gains, if I spend 20 hours mentoring one UG, they will have a much better chance of getting a good job and every person who I didn't choose to work with won't have that opportunity. Similarly with a journal, every author that's mentored and published is probably getting a straight plus to their career (academic or industry). I don't have any solutions to this, except maybe the complete dissolution of the current state of society, I just find it a difficult circle to square personally
