I've thought that for a while, but I'm not a physicist. Do you know any prominent physicists that hold that view? It seems to contradict at least the popular narrative about entropy as a property of a system.
I only have an undergraduate degree in physics, but I think the point you may be missing is that information requires some physical medium to store it.
So entropy is very roughly speaking the property of the system that determines how big a hard drive you need to store a description of that system.
"which parameters [thermodynamic variables] you know" ~ "which parameters [thermodynamic variables] you hold fixed"
(or know in average, like the energy for a system in a heat bath where the temperature is fixed)
https://bayes.wustl.edu/etj/articles/theory.1.pdf
http://nicf.net/articles/thermodynamics-statistical-mechanic...