I actually participated in the challenge for a little while and this was the approach I took before I dropped out to do a few other things.
However, what I did was a bit different -- instead of looking for a crackle, I surmised that that 'crackling' effect actually is just of course slices of the data over different rifts in the parchment, and that the data of the ink lay on the manifold of that crackling and bending.
It would not be as clear to the human eye for all of the letters, I think, as there are many, many, many layers in the scanned image, and you can only start to see a pattern emerge over time as you cycle through the images.
I was working on code that minimized an optimization function that was basically the total variance loss if I recall correctly, where it just interpolated each pixel column up and down bilinearly to 'align' the blocks of the image so that the crackle texture was flattened.
From there I planned on using a rather optimized convolutional network on the 'flattened' image, which can I think be done rather efficiently as if you look at a cross section of the scroll you can see where it's like a tree in that the pinching and such seems to be somewhat locally consistent, so you might be able to get away with some interpolation.
I should probably share the code if this is of interest to anyone, since I'm not pursuing the competition at the moment.
Also, this is why I did not buy into 3D convolutions for this, at least. Ink that has been laid and dried should follow a semi-predictable pattern that a 2D convolution can detect, I do not know if a 3D convolution really brings us anything, as the invariances we desire can be structured up front more easily.
If there is interest in the code, let me know and I can do a little digging, otherwise, it is a fun challenge, for sure.
I'd studied the problem to work on it, but didn't get as far as you. I agree that intuitively (not backed up by experiment) I expect that preprocessing to further flatten the segments and other hand-crafted features based on desired invariances should work well. It seems that a lot of people really have just used 2D convolutions, applied to just one or a few surface layers and then combined. So I'd also be interested in your code.
> I surmised that that 'crackling' effect actually is just of course slices of the data over different rifts in the parchment, and that the data of the ink lay on the manifold of that crackling and bending.
I ended up cleaning it up to share it and got sucked back into the project. >:'(((( Spent 2 hours today in RawTherapee trying to improve scan contrasts :'((((
I hope to post a perhaps more cleaned up version of at least the dataset code, the aligmnent code is extremely messy and in the discord server (licensed under 'free to steal as many ideas as you want from' basically).
Unless you've, like, named all the variables with ethnic slurs, you're not going to get in trouble because you've just copied and pasted the same subroutine twelve times instead of refactoring it into something clean.
However, what I did was a bit different -- instead of looking for a crackle, I surmised that that 'crackling' effect actually is just of course slices of the data over different rifts in the parchment, and that the data of the ink lay on the manifold of that crackling and bending.
It would not be as clear to the human eye for all of the letters, I think, as there are many, many, many layers in the scanned image, and you can only start to see a pattern emerge over time as you cycle through the images.
I was working on code that minimized an optimization function that was basically the total variance loss if I recall correctly, where it just interpolated each pixel column up and down bilinearly to 'align' the blocks of the image so that the crackle texture was flattened.
From there I planned on using a rather optimized convolutional network on the 'flattened' image, which can I think be done rather efficiently as if you look at a cross section of the scroll you can see where it's like a tree in that the pinching and such seems to be somewhat locally consistent, so you might be able to get away with some interpolation.
I should probably share the code if this is of interest to anyone, since I'm not pursuing the competition at the moment.
Also, this is why I did not buy into 3D convolutions for this, at least. Ink that has been laid and dried should follow a semi-predictable pattern that a 2D convolution can detect, I do not know if a 3D convolution really brings us anything, as the invariances we desire can be structured up front more easily.
If there is interest in the code, let me know and I can do a little digging, otherwise, it is a fun challenge, for sure.