last time I checked Google Translate was a seq2seq based model with symbol level embeddings.
The way symbol embeddings are generated doesn't lead to any real understanding of the sentence/ word/ language. It only really leads to understanding what is normally around that symbol in different situations. You could also very easily argue that the seq2seq model doesn't provide understanding, it only learns to encode the general meaning of the sequence in a fixed compressed format.
It's possible to argue that being able to compress a sequence into a fixed length vector requires understanding but I would argue that understanding requires more than just being able to do lossy compression on a sequence. In my view entity modelling and related problems are much closer to achieving some level of understanding. They at least are able to use context to figure out what parts of the sequence have what meaning and the relationship between separate parts of the sequence.
I'd be interested to hear your view of how Google Translate has understanding.
I didn't mean to say GT has understanding, quite the opposite ;) I meant to use it as an example of how translating without understanding doesn't go too well in many cases.
I think I might have misunderstood your original comment though, as we seem to mostly be on the same side of the issue. That being said, could you expand on this?:
> the seq2seq model doesn't provide understanding, it only learns to encode the general meaning of the sequence
Maybe I'm nitpicking, but "encoding the general meaning" sounds a lot like a form of "understanding", and I wouldn't say seq2seq does any of that. (That's getting pretty philosophical though...)
last time I checked Google Translate was a seq2seq based model with symbol level embeddings.
The way symbol embeddings are generated doesn't lead to any real understanding of the sentence/ word/ language. It only really leads to understanding what is normally around that symbol in different situations. You could also very easily argue that the seq2seq model doesn't provide understanding, it only learns to encode the general meaning of the sequence in a fixed compressed format.
It's possible to argue that being able to compress a sequence into a fixed length vector requires understanding but I would argue that understanding requires more than just being able to do lossy compression on a sequence. In my view entity modelling and related problems are much closer to achieving some level of understanding. They at least are able to use context to figure out what parts of the sequence have what meaning and the relationship between separate parts of the sequence.
I'd be interested to hear your view of how Google Translate has understanding.